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FOREWORD 


The  Twenty-Seventh  Conference  on  the  Design  of  Experiments  In  Army  Research, 
Development  and  Testing  was  held  In  the  Jane  S.  McKimmon  Center  on  the  campus 
of  North  Carolina  State  University,  Raleigh,  on  21-23  October  1981.  The  U.  S. 
Army  Research  Office  served  as  host  for  this  meeting.  The  Army  Mathematics 
Steering  Committee  < AHSC )  continues  to  be  the  sponsor  for  this  series  of 
meetings.  Members  of  this  committee  would  like  to  thank  Dr.  Robert  L.  Launer 
for  serving  as  Chairperson  on  Local  Arrangements,  and  Mrs.  Sherry  Duke,  who 
handled  many  of  the  administrative  details.  These  Individuals  did  an  out¬ 
standing  job  of  carrying  out  the  many  tasks  associated  with  conducting  a 
conference  of  this  size. 

Each  year  the  Program  Committee  Is  asked  to  select  Invited  speakers  who  can 
discuss  In  an  Informative  and  stimulating  manner  statistical  areas  of  current 
Interest.  At  least  one  of  the  speakers,  who  has  expertise  In  areas  of  current 
Interest  to  the  Army,  Is  asked  to  present  new  developments  In  these  fields. 

The  selection  criteria  were  certainly  met  by  the  gentlemen  giving  the  talks  In 
the  General  Sessions.  The  names  of  the  Invited  speakers  and  their  topics  are 
noted  below. 


Speaker  and  Affiliation 


Title  of  Address 


Professor  Norman  L.  Johnson 
University  of  North  Carolina 

Professor  Nozer  D.  Slngpurwalla 
George  Washington  University 


RECENT  TRENDS  IN  DISCRETE 
DISTRIBUTIONS 

ROBUSTNESS  OF  SEQUENTIAL 
EXPONENTIAL  LIFE  TESTING 
PROCEDURES  FOR  RESTRICTED  CLASSES 
OF  DISTRIBUTIONS 


Professor  Douglas  A.  Wolfe 
Ohio  State  University 

Professor  David  C.  Hoagl In 
Harvard  University 


COMPARING  SEVERAL  GROUPS  IN  A 
TWO-WAY  LAYOUT  SETTING 

APPLICATION  OF  EXPLORATORY  DATA 
ANALYSIS  TECHNIQUES  IN  MORE 
COMPLEX  MODELS 


Professor  Walter  L.  Smith 
University  of  North  Carolina 
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In  addition  to  the  two  Invited  addresses  on  the  first  day  of  this  meeting, 
there  were  five  solicited  talks  on  product  assurance.  These  were  delivered  by 
Army  scientists  that  are  specialists  In  this  area.  Another  event  associated 
with  this  meeting  was  a  tutorial  seminar  on  "Quality  Control".  It  was  held 
just  preceding  the  conference  on  19-20  October  and  was  given  by  Professors  P. 
M.  Share  and  D.  R.  Jensen  of  the  Virginia  Polytechnic  Institute  and  State 
University.  This  course  gave  the  standard  procedures  for  monitoring  a  process 
by  variables  or  by  attributes  In  the  case  of  a  single  quality  characteristic. 

The  winner  of  the  first  Wilks  Award  for  Contributions  to  Statistical 
Methodologies  In  Army  Research,  Development  and  Testing*  was  presented  to 
Professor  Robert  E.  Bechhofer  of  Cornell  University  at  a  luncheon  on  the  first 
day  of  the  conference.  He  richly  deserves  this  honor  for  his  many  scientific 
contributions  to  ranking  and  selection  procedures  as  well  as  other  statistical 
afeas.  He  Has  given  freely  of  his  time  to  help  Army  scientists  develop 
statistical  skills.  Recently  he  solved  a  very  Important  problem  In  ballistic 
testing  related  to  kinetic  energy  penetrators. 

Members  of  the  AMSC  would  like  to  take  this  opportunity  to  express  their 
thanks  to  Mr.  Philip  G.  Rust  of  Thomasvllle,  Georgia  for  endowing  both  of  the 
Wilks  Awards.  His  generous  gifts  In  memory  of  his  friend,  Sam  Wilks,  will 
contribute  to  the  welfare  of  the  military  services  as  well  as  foster 
statistical  science  In  general. 


For  more  Information  on  this  award  see  "Open  letter  on  the  New  Wilks  Award7 
It  Is  printed  following  this  foreword. 
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OPEN  LETTER  ON  THE  NEW  WILKS  AWARD 


TO:  Dr.  Jagdlsh  Chandra,  Chairman  of  the  Army  Mathematics  Steering 

Committee  (AMSC) 

FROM:  Dr.  Robert  L.  Launer 


This  letter  Is  being  written  as  a  result  of  recent  developments  con¬ 
cerning  the  establishment  of  a  new  Wilks  award.  The  reason  for  bringing 
this  matter  to  your  attention  will  become  clear. 

The  Samuel  S.  Wilks  Memorial  Medal  and  Award  was  initiated  In  1964  with 
a  $5000  gift  from  Mr.  Philip  B.  Rust  of  Thomasvllle,  Georgia.  This  award, 
presented  each  year  at  the  Army  Design  of  Experiments  Conference,  has  been 
coordinated  and  administered  by  the  American  Statistical  Association, 
because  of  the  difficulties  Involved  at  that  time  In  administering  the  funds 
Internally.  Mr.  Rust  Intended  that  this  award  be  given  to  a  statistician 
for  contributions  to  Army  technology.  Unfortunately,  he  did  not  compose  a 
specific  citation  which  would  be  used  as  a  guide  in  choosing  the  annual 
winner  of  the  award.  A  complete  record  of  the  ceremonial  remarks  made  at 
the  time  of  the  Initial  award  are  given  In  the  proceedings  of  the  Tenth 
Design  of  Experiments  Conference, 

Questions  have  been  raised  about  some  of  the  recent  winners  of  the  Wilks 
award.  These  were  communicated  to  Dr.  Ralph  Bradley  of  Florida  State 
University  and  President  of  the  American  Statistical  Association.  As  the 
result  of  the  ensuing  protracted  conversations,  Mr.  Rust  made  another  gift 
for  the  establishment  of  a  second  Wilks  award,  and  to  augment  the  endowment 
for  the  original  Wilks  award. 

The  Army  Gifts  Fund  Office  {AGFO),  under  the  Office  of  Secretary  of  the 
Army  (AR1-100),  acts  as  the  custodian  of  gifts  to  the  Army.  In  the  case  of 
monetary  gifts  the  AGFO  channels  the  funds  through  the  commander  of  an  Army 
Installation  having  close  ties  to  the  donors  expressed  purpose  of  the  award. 
Since  the  Wilks  award  Is  for  academic  or  scientific  excellence,  the  Adjutant 
General  of  the  Army  has  established  that  the  Army  Research  Office  (ARO)  can 
serve  as  such  an  Installation.  The  Commander  of  ARO  has  agreed  to  have  the 
funds  channeled  through  his  Finance  and  Accounting  Office  and  to  have  the 
other  aspects  of  this  award  handled  by  the  AMSC,  an  Intra-Army  committee. 

In  order  that  the  new  Wilks  award  begin  on  a  firm  basis,  with  the  first 
award  given  In  1981  If  possible,  and  to  avoid  unforeseen  difficulties  and 
criticisms,  It  Is  proposed  that  the  new  Army  Wilks  award  be  administered 
In  accordance  with  the  following  points. 

1)  The  award  should  be  called  "The  Wilks  Award  for  Contributions  to 
Statistical  Methodologies  In  Army  Research,  Development  and  Testing". 
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2)  The  award  winner  should  be  chosen  by  a  majority  vote  of  an  ad-hoc 
committee  of  five  statisticians.  The  committee  should  consist  of  at  least 
two  statisticians  from  the  Army  community  and  at  least  two  statisticians 
from  academia,  each  serving  for  two-year  terms.  The  Initial  committee 
should  be  appointed  by  the  Chairman  of  the  AMSC.  Thereafter,  the  subcommit¬ 
tee  on  Statistics  and  Probability  shall  annually  nominate  one  new  member. 

The  winner  of  the  award  may  replace  one  member  of  the  selection  committee 
for  the  following  year.  The  Chairman  of  the  AMSC  may  act  as  a  non-voting 
chairman  of  the  selection  committee,  or  he  may  appoint  a  representative  to 
act  In  his  stead  (voting  or  non-voting). 

3)  The  Chairman  of  the  selection  committee  should  convene  the  selection 
committee  early  enough  that  the  award  can  be  made  at  the  annual  Design  of 
Experiments  Conference. 

4)  Nominations  of  candidates  for  the  award  may  be  forwarded  to  any 
member  of  the  selection  committee.  Solicitations  may  also  be  made  through 
announcement  letters  for  the  annual  DOE  conference,  or  any  other  appropriate 
method. 

5)  The  chairman  of  the  selection  committee  should  serve  as  the  coor¬ 
dinator  of  the  award  money.  This  should  be  In  the  form  of  a  check  made 
payable  to  the  winner  of. the  award. 
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RECENT  DEVELOPMENTS  IN  DISCRETE  DISTRIBUTIONS 
N.L.  Johnson 

University  of  North  Carolina 
Chapel  Kill (North  Carolina 

1.  Introduction.  In  accordance  with  the  title  of  thla  talk,  I  plan  to 
give  a  (very)  concise  description  of  work,  (very)  nearly  all  on  discrete 
distributions,  published  in  the  period  1969-81.  This  seems  a  rather 
strange  topic  for  a  "keynote"  speech,  so  I  will  provide  some  explanation. 

My  major  interest  is  to  illustrate  acme  personal  attitudes  towards 
research  results.  Many  fields  could  serve  as  sources  of  illustrative 
material.  I  have  chosen  discrete  distributions  because,  for  about  the 
last  two  years,  Dr.  Samuel  Kotz  and  I  have  been  collecting  material  on 
this  topic  for  an  article  for  the  International  Statistical  Review  (XSR). 
So  it  is  a  field  wherein  I  feel  at  home,  and  some  of  the  illustrative 
material  may  be  of  interest  to  aoma  of  you,  even  if  ny  carmen  ts  are  not. 

The  bibliography  for  the  article  contains  nearly  700  entries,  so  you 
will  not  be  surprised  to  learn  that  the  Editor  of  the  ISR  has  asked  us  to 
reduce  the  length  of  the  article  to  about  20%  of  the  original  (though 
he  has  generously  agreed  to  publish  the  vtiole  of  the  bibliography).  So 
there  is  (for  ma)  a  second  attraction  -  the  opportunity  to  draw  attention 
to  a  few  matters  vhich  will  be  mentioned  inadequately  -  if  at  all  -  in  the 
published  article. 
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In  order  to  make  it  possible  to  cover  a  considerable  amount  of 
material,  I  will  make  use  of  a  number  of  summaries  in  note -form,  of  vrtiich 
you  should  have  copies.  Much  of  my  talk  will  consist  of  Garments  on 
these. 

The  collection  and  classification  of  the  material  was  done  jointly 
by  Dr.  Kotz  and  myself,  but  all  expressions  of  opinion  are  my  own. 

2.  Prejudices .  Before  coming  to  details,  a  few  words  about 
prejudices,  it  is,  unfortunately,  true  that  many  of  us  -  most,  I  suppose 
-  sometimes  react  negatively  to  the  mere  title,  or  general  content  of  a 
report  or  paper.  (Let  me  admit  to  an  unreasoning,  and  unreasonable 
hostility  to  papers  on  characterisations  and  on  Least  Squares  -  plain  or 
modified) .  He  may  be  influenced  by  fashion  -  a  need  to  appear  in  accord 

i 

with  currently  dominant  prejudices  -  or  we  may,  in  our  own  experience, 
have  found  the  topics  to  be  unrewarding  and  so,  by  some  sort  of 
extrapolation  decide  they  are  of  little  importance  generally. 

Here  are  two  examples  of  ways  in  which  prejudice  can  arise  -  the 
first  reflecting  types  of  attitude  which  are  still  not  unknown;  the  second 
relevant  to  our  present  topic. 

1)  (R.  and  Peto,  replying  to  discussion  of  their  paper 

"Asymptotically  Efficient  Rank  Invariant  Tast  Procedures"  -  J,R.  statist. 
SgCj.  135A  (1972),  p.  205). 
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"Every  analysis  o£  survival  data  should  either  be 
efficient  against  Lehmann  alternatives  or  have  a  very 
clear  reason  for  not  being  so;  the  Lehmann  family  is  the 
"normal"  distribution  of  survival  theory." 

(My  italics.  No  evidence  for  the  last  statement  is  presented.) 

ft  more  balanced  (and  honest)  approach  is  provided  by  N.  Mantel  in 
"Evaluation  of  Survival  Data..."  in  Cancer  Chemotherapy  Reports, 50, 
(1966)  p.  167. 

"It  is  unlikely  that  in  any  real  instance  in  which  the  two 
force -of  -mortal i  ty  functions  Z^t)  and  Z2(t)  differ  that  any 
simple  relationship  exists  between  them.  In  principle,  however, 
it  is  possible  to  determine  the  power... for  alternatives  - 
Zj_(t>  *  kZj(t),  k  f  1."  (These  are  Lshmann  alternatives.) 

2)  Prom  a  review  by  E.J,  Williams  of  RCSW.  Vols.  2/3  (see 
References  -  III)  in  Australian  J,  Statist..  (1973),  p.  183i 

"After  reading  some  of  the  contributions,  this  reviewer 
would  question  whether,  at  this  stage  of  development  of  the 
statistical  art i 

(1)  finding  a  distribution  that  fits  well  to  data  is  a  contri¬ 

bution  to  statistical  science,  and 

(2)  the  study  of  a  particular  form  of  statistical  distribution  is 

a  contribution  to  statistical  theory." 
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I  am  not  claiming  that  thasa  authors  vara  unjustified  in  their 
specific  comments ,  but  X  do  think  that  their  modes  of  expression  can  lead 
to  development  of  prejudice. 

There  will  be  plenty  of  prejudice  in  this  talk.  As  a 
counterbalance,  it  is  well  to  keep  in  mind  that! 

(a)  The  fact  that  one  does  not  understand  something  is  evidence 
neither  of  its  value  or  its  lack  thereof,  though  it  may  reflect  lack  of 
skill  in  exposition. 

(b)  It  is  lasy  to  judge  a  paper  by  its  title  alone,  or  a  piece  of 
work  by  its  field. 

(c)  In  judging  "applicability,"  it  is  important  to  consider  as  many 
aspects  of  application  as  possible  and  not  just  those  based  on  personal 
experience,  however  lengthy  or  distinguished. 

On  the  other  hand, 

(d)  The  appearance  of  a  large  number  of  papers  on  the  same  topic  in 
a  brief  period  of  time  may  not  signify  that  the  topic  is  of  lasting 
importance.  It  may  just  mean  that  it  appears  to  offer  the  proepect  of 
quick  results  from  small-scale  investigations.  (This  is  often  the  genesis 
of  a  "fashion"),  and  finally, 

(e)  It  is  a  salutary  exarcise  to  read  copies  of  journals  from  10  or 
more  yaars  ago  and  to  try  to  assess  the  reasons  why  ths  articles  contained 
therein  were  accepted  for  publication. 
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3 .  Familial  (Systems,  Classes,  ate.)  of  Distributions.  The 
construction  of  new  families  of  distributions  still  fascinates  a  number  of 

workers,  nyself  among  them.  Age  has  tempered  some  earlier  enthusiasm,  and 

* 

I  am  now  more  aware  that  usefulness,  more  than  novelty,  is  the  essential 

property  of  value.  "Unefulness"  can  be  interpreted  quite  broadly  but  a 

new  family  should  not  be  boo  similar  to  an  existing  one  unless  it  is  also 

sinpler  in  sons  important  respects. 

I  believe  there  ija  real  value  in  suitable  systems  -  particularly 

because  they  can  assist  appreciation  of  relationships  among  distributions , 

The  requirement  of  suitability,  however,  is  important.  The  mere 

variety  and  number  of  distributions  inoluded  in  a  system  are  not,  in 

themselves,  measurea  of  its  importance  -  still  less,  of  its  practical 

value.  (For  example,  the  class  defined  by  l?v  -  1  ia  very  broad  but 

x  * 

contributes  nothing  to  understanding.)  What  matters  is  the  inclusion  of 
as  wide  a  variety  as  possible  within  as  specific  a  formulation  as 
possible. 

(We  now  refer  to  Table  la.)  An  outstanding  exanple  is  the  class  of 
power  aeries  distributions  (PSD).  In  fact,  it  is  only  the  $  part  that 
stops  them  being  uselessly  general,  but  this  is  enough.  A  competitive  - 
and  complementary  -  system,  the  factorial  series  distributions  (PSD) ,  has 
been  introduced  in  our  period. 


3 


>1. .  .  .»  .,,.1.1 


Generalized  hvpar geometric  aeries  distributions  have  explicitly 

structured  b(x)  functions.  The  class  is  very  broad.  Dacey  (1972)  lists 

some  50  members  of  the  class,  utilising  values  of  h  and  k  not  exceeding  3. 

In  practical  use  it  is  rare  to  have  either  h  or  k  even  as  great  as  3. 

For  one  thing,  there  are  (h+k+1)  parameters  and  Occam's  Razor  is 

well-established  in  distribution  construction.  ("Fashion"  and/or 

"Prejudice"?)  For  another,  if  wa  consider  the  quantities 

h  K 

Vi  ■  > 

we  see  that  (i)  if  h  and  k  are  not  known,  it  is  likely  to  be  difficult  to 
estimate  them, 

(ii)  even  if  h  and  k  are  known,  values  of  Ux  for  quits  an 
extensive  range  of  values  of  x  will  be  needed  to  estimate 
6,  a^s  and  b^'s. 

Even  supposing  9-1  (generalized  hvparqeamstrlc  distributions)  it  is 
very  doubtful  tfiether  much  is  gained  by  increasing  h  and/or  k  above  2. 

The  use  of  Ux  (and  similar  functions)  to  systematize  search  for  an 
appropriate  distribution,  has  much  to  oommend  it.  Ond  (1972)  has  provided 
rules  for  using  this  approach  when  h  -  1  and  k  -  0  (so  that  CJX  is  a 
linear  function  of  x).  (See  Table  lb.) 
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It  in  easy  to  see 

(1)  how  (in  principle)  this  approach  can  be  extended  to  higher 
values  of  h  and  k,  and 

(ii)  the  considerable  technical  difficulties  likely  to  arise. 

Despite  (ii),  I  think  that  this  kind  of  approach,  with  suitable 
(approximate)  allowance  for  sanpling  variation,  can  be  of  real  value.  It 
can  also  provide  a  way  of  assessing  when  two  (or  more)  distributions  - 
however  attractively  named  ~  are  likely  to  be  indistinguishable  in 
particular  kinds  of  applications. 

The  modified  power  series  distributions  (MPSD)  are  extensions  of  the 
PSD  in  the  sanse  that  when  h(6)  is  invertible  they  are  just  PSD's.  By 
taking  h < e  >  •  e/g(e )  the  interesting  Lagrange  distributions  are 
obtained.  These  use  the  Lagrange  expansion 

f(B)  -  f CO)  +  l  Dj“1{[G(t)]jf)(t)) 

j-1  V  t-0 

for  the  p.g.f.  f(a).  G(s)  is  also  taken  to  be  a  pgf  with  g'(0)  <  1. 
Then 

P0«f(0),  Px  *  JT •DK"1{[G(t)]*f'(t)} 

t-0 
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A  table  from  Consul  and  Shenton  (1972)  shows  a  nuirber  of  special 

cases. 

Assessment  of  the  "value1*  of  this  class  of  distributions  needs  some 
careful  balancing.  There  are  useful  general  formulae  for  moments,  in 
terms  of  cumulants  corresponding  to  f(»)  and  G(  •).  It  is  possible  to 
approximate  binomial  and  negative  binomial  distributions  quite  well  with 
the  Lagrange  double  Poisson  (e.g.  Jain  (1974)). 

On  the  other  hand,  the  formal  development  does  not  as  yet  provide 
(for  me)  a  very  helpful  background  to  comprehend  the  nature  of  the 
distributions.  Some  of  these  do  arise  "naturally"  in  queueing  theory 
(Shenton  and  Consul  (1973),  Kuimr  (1981))  and  in  ballot  theory  (Narayana 
(1979)). 

A  feature  of  these  distributions  is  that  it  is  often  not  Immediately 
obvious  (to  me)  that  l  Px  ■  1.  For  the  "double  binomial,"  for  exairple, 
this  relationship  can  be  derived  from  Abel's  identity.  There  are  related 
identities  (e.g.  Riordan  (1979))  from  vrtiich  other  distributions  can  be 
concocted  in  a  formal  way.  (See  Table  lc).  Whether  they  are  all  Lagrange 
distributions  I  do  not  know.  In  fact,  I  do  not  know  a  general  method  for 
deciding  whether  a  given  distribution  is  a  Lagrange  Distribution.  (The 
"practical  value"  of  auch  knowledge  is  at  present  uncertain  -  prejudice 
may  auggeat  it  would  be  only  of  intellectual  value  -  but  it  would  interest 
me. ) 
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4.  Modifications.  Now  we  look  at  Table  2.  There  has  been  considerable 
interest  in  "monkeying  about"  with  distributions.  A  very  sinple  form  is 
just  increasing  (or  decreasing)  one  probability  (usually  Pq)  and 
decreasing  (or  increasing)  all  the  others  proportionately  -  giving  rise 
to  "Inflated"  (or  "deflated")  distributions.  Truncated  distributions  are 

t 

limiting  cases . 

If  the  adjustments  are  proportional  to  the  original  ?x  values  we 
have  weighted  distributions  with 

V '  "xV'J  Vs-1  ■ 

Particular  cases,  such  as  wx  a  linear  function  of  x,  so  that 
PXVPX  ■  a  +  3x  are  of  interest,  because  one  can  fit  a  standard  {Px> 
to  data  (fx)  and  then  study  the  ratios  l*x/Px>  a*  a  function  of  x, 
which  nuy  lead  to  a  suitable  and  sinple  modification. 

"Mixing"  (or  "compounding")  is  a  well-established  form  of 
modification.  The  table  shows  the  notation.  Note  that  F^  ^  P2  is 
usually  called  a  Fj-Fj  distribution  (though  sometime  it  ie  called 
F^-Fj).  By  choosing  different  pairs  of  F^Fj'a  quite  a  wide  range 
of  distributions  can  be  obtained.  Seme  relatively  recent  exaitplee  are 
described  in  the  references.  While  interesting  discoveries  can  be  made 
trom  speculative  FlfF2  Payings,  it  is  usually  more  attractive  when 
the  mixing  arises  from  a  natural  model.  The  subclass  of  oonpound  Poisson 
(l.e.  with  F^  Poisson)  has  been  especially  popular.  There  is  a 
respectable  reason  for  this  (i.a.  apart  from  mathematical  sinplicity). 
Poisson  corresponds  to  "independence  in  time  and/or  space")  ccnpound 
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Poisson  can  be  detected  by  comparing  data  with  a  fitted  Poisson 
distribution.  (See  e.g.  Shaked  (1980).) 

We  note,  in  passing,  that  "Random  Sum"  or  "generalization " 

(Pl  v  F^5  can  be  expressed  as  a  mixture  of  convolutions 

F|Na  Fj  {"Fj-generalization  of  Fi") 

Of  Fr 

New  distributions  can  sometimes  be  found  as  limits  of  "old"  ones. 
Recently  Sibuya  (1979  )  obtained  dioamma  and  trigamma  distributions 
as  limits  of  a  zero-truncated  inverse  Polya-Eggenberger  distribution. 
These  are  just  special  sorts  of  hypergeometric  distributions;  of  special 
interest  because  of  their  relation  to  logseries  distributions. 

Mora  sweeping  modifications  can  lead  to  very  broad  systems.  Ihe 
Poisson  modification  of  Bernoulli  trials  (allowing  success  probability 
to  change  from  trial  to  trial)  which  leads  to  Poisson  binomial  (bo  be 
distinguished  from  Poisson-binomial  and  binomial-Poisson)  distribution  can 
be  modified  by  allowing  for  dependence  between  trials  (see  Table  2).  The 
references  given  develop,  inter  alia,  (joint)  distribution (s)  of  total 
number (s)  of  successes.  I  find  these  studies  of  more  use  for  the  general 
nature  of  the  results  than  for  their  specific  forms. 

5.  Damage  Models  and  Characterizations.  I  have  already  admitted  to  a 
negative  attitude  to  work  on  characterizations.  This  is  based,  I 
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believe,  on  a  feeling  that  characterization  depends  on  a  distribution 
being  followed  exactly;  the  nature  and  amount  of  departure  from 
characterization  corresponding  to  given  departure  from  the  assumed 
distribution  seems  to  be  of  little  account.  It  would  be  an  encouraging 
sign  if  characterization  research  were  to  be  accompanied  by  some 
indication  of  robustness. 

A  rather  prominent  example  of  the  baleful  influence  of 
characterization  is  in  the  study  of  "damage  models."  In  the  simplest 
form,  these  models  include  three  random  variables  X,  Y  and  Z  with  X  »  Y  + 
Z;  X  represents  "undamaged"  value,  Y  represents  observed  ("damaged") 
value,  and  Z  represents  the  "damage."  (X  might  be  the  nunber  of 
nonconforming  items  in  a  sample;  Y  the  number  detected  by  an  inspection 
process;  then  the  "damage,"  Z,  is  the  number  of  nonconforming  items  not 
detected  in  the  inspection.) 

In  1968,  Rao  and  Rubin  shoved  that  if,  Y  and  Z  are  independent  and 
the  conditional  distribution  of  Y,  given  X  is  binomial,  then  Y  and  Z  each 
have  Poisson  distributions.  Since  then,  there  have  been  many  variations 
on  this  theme  (see  Table  3) ,  with  the  common  feature  that  Y  and  Z  cure 
mutually  independent.  Is  this  assumption  likely  to  be  realistic?  I  have 
not  seen  any  investigation  of  this. 

A  related  topic,  which  does  seem  to  have  sane  virtue,  was  studied 
by  Samaniego  (1976).  He  defined  "convoluted  Poisson"  variables 
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<Y  *  +  Xj  where  and  are  independent  and  X^  is  Poisson) 

and  finds  characterizations  of  the  distribution  of  Y.  It  is  suggested 
that  Y  might  represent  an  "overcounted"  Poisson  variable. 

6.  Approximations .  Before  the  advent  of  powerful  conputing  aids, 
approximations  were  useful  because  tiiey  made  calculations  feasible  vtoich 
would  otherwise  have  been  impossible.  One  might  expect  that  with  the 
present  profusion  of  conputing  power  there  would  be  decreasing  interest  in 
approximations.  One  would  be  wrong.  Just  considering  approximation  of 
the  tail  probabilities  of  binomial  by  those  of  unit  normal  distributions 
we  have  the  quits  imposing  list  shown  in  Table  4,  all  published  in 
1968-80.  similar  tables  can  be  constructed  for  approximations  of  Poisson 
and  hypergeometric  distributions. 

There  are  varied  reasons  for  these  phenomena.  Of  course,  many  of 
the  computer  programs,  themselves,  use  approximations.  Some 
approximations  (such  as  that  of  a  normal  by  a  lognormal  or  logistic 
distribution)  can  be  used  to  simplify  more  extensive  theoretical  analysis. 
Sometimes,  also,  it  is  useful  to  have  a  quick  wav  of  calculating  an 
approximate  value.  Their  major  value  (in  my  opinion)  is  their  ability  to 
present  easily  comprehended  pictures  of  ttfiole  sets  of  results. 
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Xn  addition  to  these/  more  or  less  valid,  reasons,  we  are, 
unfortunately,  left  with  the  inpresaion  that  some  approximations  have  only 
their  elegance,  and  some,  even,  only  their  novelty,  to  recommend  them. 
As  a  general  rule  the  approximation  should  be  rather  Less  complicated  than 
the  quantity  being  approximated.  This  does  not  seem  to  be  true  for  some 
items  in  ‘Table  4. 

Sometimes  it  is  not  entirely  clear  in  which  direction  the 
approximation  is  most  useful.  Fbr  example  the  Lagrange  double  Poisson 
with  parameters  ■  N{  <  l+P^^-d+P)""1/2) ,  ■  1-(1+P)-1/2 

and  the  negative  binomial  with  parameters  N,P  are  very  similar  to  each 
other.  Jain  (1974)  implies  that  the  latter  is  e  useful  approximation  to 
the  former,  yet  many  persons  would  consider  the  negative  binomial 
"simpler"  than  the  double  Poisson. 

Recently  an  increase  in  research  on  accuracy  of  established 
approximations  has  been  a  welcome,  though  often  tedious  and  rarely 
elegant,  feature  of  statistical  literature.  It  is  to  be  hoped  that  some 
way  will  be  found  of  presenting  the  results  of  such  research  in  both  more 
digestible  and  more  permanent  forms  (e.g.  monographs),  oambining 
attractive  production  with  careful  non-partisan  effectiveness. 
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7.  Concluding  Remarks.  You  may  still  feel  that  I  have  chosen  an 
unfortunate  topic  as  a  vehicle  for  these  remarks  on  evaluation  of 
research.  I  do  not  think  this  is  so  Taking  the  risk  of  undue  repeti¬ 
tiveness  ,  I  believe  similar  assessments  would  be  reached  in  very  many,  if 
not  all  fields,  especially  if  one  discounts  the  effect  of  current, 
transitory  fashions  with  their  new-found  but  often  ill-founded  en thus lane . 

1  think  that  we  also  should  allow  for  the  tendency  (perhaps 
unconscious)  to  welcome  the  idea  that  much  research  (in  other  fields  than 
one's  own,  and  even  in  one's  own  field,  by  other  workers)  is  of  little 
permanent  value,  so  that  there  is  no  need  to  spend  time  and  effort  in 
understanding  it.  It  is  undoubtedly  true  that  the  present  organization  of 
research  effort  in  the  world  is  such  that  there  is  much  waste  -  both  in 
redundancy  and  in  publication  piecemeal  of  special  results  of  little 
intrinsic  interest  or  value  except  "novelty."  But  this  does  not  mean  that 
we  should  underestimate  the  value  of  all  that  is  published.  We  still  tend 
to  start  with  an  overoptimistic  idea  of  how  much  to  expect  from  a  piece  of 
research,  quickly  followed  by  disillusionment  when  reality  does  not  match 
up  to  our  preconceptions.  We  should  try  to  attain  a  realistic  view  of 
what  to  expect  of  "good"  research.  I  suggest  this  means  that  evaluation 
has  to  be  delayed  for  a  few  years  (3  or  more,  perhaps)  to  see  more  clearly 


where  the  research  results  stand  in  relation  to  the  general  development  of 
a  subject. 

In  the  specific  context  of  the  present  talk  we  can  say,  I  think, 

that 

(1)  Mark  on  new  families  of  discrete  distributions  has  increased 

(i)  our  power  to  construct  useful  mathematical  frameworks 

and  (ii)  our  ability  to  appreciate  relationships  between 
different  frameworks. 

(2)  Modification  of  distributions  has  been  systematized,  and  its 
possibilities  are  becoming  more  clearly  realized. 

( 3 )  The  power  and  usefulness  of  approximations  have  been  Increased 
and  is  now  more  generally  appreciated,  and  msthods  of  assessing  the 
accuracy  of  approximations  are  becoming  batter  understood. 

( 4 )  Although  much  of  the  development  of  new  multivariate  discrete 
distributions  has  been  rather  formal,  there  is  a  slow  growth  in 
appreciation  of  the  kinds  of  such  distributions  now  available  to  the 
analyst. 
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SYSTEMS  OF  DISCRETE  DISTRIBUTIONS  -  I 


(I)  (II)  POWER  SERIES  (PSD(6ig(e))) 


P  ■  •  — 

x  xT  g 


8X  .  bfx 


(x*0  »!»•••) 


b(x)  -  DXg(6)  |Qll0 
psD*n(6jg(e))  ~  PSD(0i(g(e)}n) 


FACTORIAL  SERIES  fPSO(Nlh(N))) 

px  ■  hr  '  k$}  (x'0,1m,,,N) 

c(x)  -  Ax h(N)|, 


>N»0 


AXh(0) 


(i)'  cm* 


Ml jiUteialt 

*< 

m  0,J 

p  -  {  n  -M  b(g)/g(ft) 

*  j.l  *j'- 

«*>  *  o“l>  l«lw 


C*j) 

.  n  N4  j 

p  ■  { n  -J-p}  c(a)/h(M) 

»  j-1  V 

e(£)  “  (  ^  An|)  h(8) 


(III)  MODIFIED  POWER  SERIES  (MPSD  (h(B)}  g(8)) 


%  •  i!^T1L  1  I®-. 


(IV)  LAGRANGE  CLCGCO .  ft*))  (from  MPSD  with  h(B)  ■  6/G(6)) 


^  j 

Probability  generating  function  (pgf)  is  f(s)-f(0)  +  l  l*/-9 (D^  ^(t))^  f  (t)>  lt-Q]  • 

,  iml 


Usual  to  take  G(»)  also  a  pgf  with  |G'(0)|  <  1 


P0  ■  f(0)5  Px  ■  Ox“1{[C(t)l*f'(t)HtIl0  (x-1,2,...) 


(V)  GENERALIZED  HYPERGBOMETRIC  SERIES  (hGkHypg  (g,ib*e)) 

h  ^ 

p» "  n  *|x3]/[  n  bjx]]}»(ex/xt)  (x-o.i,...) 

x  n  r  i  j 

(If  6  -  1  ->  GENERALIZED  HYPER GEOMETRIC) 


(VI) 


v 


[tttx]/c[*J][  5  {b[X,10Xj/*il>] 

j-1  >  } 


m 

(x  -  l  X,i  x,*0,l,2, . . .) 
j-1  i  3 
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(VII) 


JJU'.I.I  i.  rlf  K'.'V  («l 


FM.—tmv.i  |irw:»ft{»vj  rw?r 


[oi]  [S]  [x]  [xl  [ct]  [3] 

LIMITS  or  INVERSE  P^LYA-EGGENBERGER  (TK(0)):  l>  *  lr-4. — - - £ - rrh.1.,rX.T  )~l 

X  Y  xl (a+0+Y)  y1^^' 

e  +  0  leads  to  JliGAMMA;  P^  ■  t<p(oi+Y)  -  «HY)  >"1  «^/(  (a+y)  ^xl  (x-1,2,...) 

P  +  0,  a  +  0  leads  to  1H1 GAMMA;  -  {*'(y)}'1(*-1)I/(yWx)  (x-1,2,...) 

[xjl 

INV.  PoW-EGGENBERGBR  (TR  ((>) ) :  f%.  ) 

m  n 

(<*  “  l  uij }  x  ■  l  x,j  x.  *  0,  X  ■  excluded) 

J«1  J  j-1  3  J 


[Xj] 

0  +  0  loads  to  MULTIVARIATE  OIGAMMA;'  P  -  (<i(a+Y)  -  i^(Y}  >-1  It  (4-rJ 

x  (a+y)1  Jj-1  jl 

m 


3  +  0,  01,+  *,  Y  +  *  with  a./(a+Y)  ■  0,  and  J  6.  ■  6  loads  tD 
3  3  •  3  j-1  3 

MULTIVARIATE  LOGSERIESi  l’x  ■  n  (fljJ/xj) 

j-1 

[0  +  0,  a^j  +  0  with  a^/a  ■  tj ,  y  fixed  gives  a  degenerate  distribution] 
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SYSTEMS  OF  DISCRETE  DISTRIBUTIONS  -  II 


choice  ol 


(a-x)P  . 

CVHI)  P  -  P - £±-~ 

b0+blX4b2x 


x  P. 


If  plot  of  against  x  is  linear  then  bj  ■  0  and  distribution  is  indicated  by 

proportios  of  tho  linear  rolation  U  ■  x  +  (a-x)/b. 

x  l 


INTERCEPT  ("U0") 


♦  >  0 
(n+l)p/q  >  0 
(N-1)P/Q  >  0 
-<t>  <  0 


SLOPE 


■p/q  <  0 
p/q  >  o 
<p  >  o 


DISTRIBUTION 


Poisson  (40 
Binomial  (n,p) 

Negative  Binomial  (N,P) 
Logseries  (cp) 

Discrete  rectangular 


(^gxj^jtoiaUaatXr 


(POISSON  BINOMIAL] 


(IX)  P,  «  P 


i  •  P.  i  ,  -  Pr (  n  (X,-x,)]  (l,  -  0,1) 

m  _ 

.dditive  system:  p  /{  n  I'  )  ■  £  £{p,  ,  / (p ,  p.  )}  -  Q  +  1 

~  j-1  xj  n<l>  a  b  *a  1b  4 


Additive  sys 


Multiplicative  system:  P,  ■  H  ■  \ 

~  a<b  1a1b 


C*01  " 


m 

Myltlyqrlatp  (pgf  of  multinomial  is  (  l  p,s.) 

j-1  •*  J 


11  1  m  a 


MULTINOMIAL  BINOMIAL 


■  nt  ( 1  I  !  P  n  ,,j)"  [J...Ips 

V°V°  V0*!'1 


MULTINOMIAL  MULTINOMIAL:  pgf  is  as  above,  but  somo  (at  least)  of  the 
ah's  have  uppor  limits  greater  than  1. 
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» 

(XI)  "ABBL"  DISTRIBUTIONS 

3UASI- BINOMIAL  I:  -  a(«^nT)‘n(|J),(«+tx)x“I{Bn(n.x)}n“x 

QUASI- BINOMIAL  II!  PJt-aB{c»+B)‘1(otg+nT)'(n'1JcJ)(a+Tx/l'1{Bi-T(n-x)?1‘x‘1 

(x  ■  0«  1 1 . . , ,n)^ QUASI-HYPBRGEOMRTRIC :  Px«  aB(a+0+nT)(^3)'l{B+r(n-x))‘l 

QUASI-POLYA:  Px“[(a*S+nT)tnl]*1u('’){a+TCx-l)^n'1^T(n.x))^n-x^  . 


QUASI-MULTINOMIAL  I:  P 


r  n  m  \  m  , 

‘JiV®1  p . vn'JI1Xj/(A“j (vv,J’' 

Is"  I  xj)  1  J  (0<x  :  £  yn) 
j-1  J  J  j.l  J 


table  ?  moo iimcation  mi:ti tons 


(XII)  a)  INFLATION  AND  PUT l<AT J ON 


p*  -  p  *  o}  p*  -  p*o-pv  -co/u-p,  )  (x^Ai'F,  *  a  *  i-p* ) 
x0  x0  x  *  "0  *0  ‘  *0  0 

.0  >  0  ■>  inflated  distribution;  a  <  0  «>  deflated  distribution 

a  ■  -P_  “>  tnmoatad  distribution  (ospoclally  with  x„  ■  0) 

*0  " 

Xq  ■  0,  0  >  0  »>  'addod  ttroos'  distribution 

M-  •  (1-t 'bK  *  T$-  *0 


(xiii)  b)  weigiiting 


PS  '  wxV|wjpj 


(wx  *  0) 


(XIV)  c)  MIXING  (Soma  oxsmplos  of  COMPOUNlJ  POISSON  Poi  (Cl)  a  G-  "G-POISSON") 

~  e 

INOTF:  POISSON- BINOMIAL  is  Bln(n,p)  a  Fol (0)] 

n/k 

(XV)  POWER  FUNCTION- POISSON  Poi (6) a  Powar  function  | 


.  (XVII) 


LOGNORMAL- POISSON 


INVERSE  GAUSSIAN- POISSON 


Poi (6) a  Powar  function  (h,>) 
0 

Poi(O)  a  N(u,o2) 

1or6 

Poi (0)  a  Gauss'1 (m,o2) 


(XVIII)  ? 

(XIX)  (XIX)'  BESSEL  A. -POISSON,  BESSEL  B-P01SS0N 

(XX)  NEGATIVE  BINOMIAL-POISSON 


BINOMIAL- POISSON 


BETA-BINOMIAL  Bin(n,|>) a  Bata  (a,@) 

P 

(XXI)  Ganoraliiod  by  taking  p  ■  g(c)  with  c  ^  Bota(a,0) 


Poi(O)  a  -Bata (a, S) 
1-e'6 


Poi (0)  a  Nog.Bin(N.P) 

6 /* 

Pol (9)  a  Bin (n ,p) 

6/d» 


FSD(N;(h(N)}n)  a  PSO(0;g(O)) 
n 


(XXII) 


('snowball  sampling') 


RANDOM  SUM  is  a  special  sort  of  mixing 


F* 


N 


A  G , 

N 


(also  called  'G-general i  ccd  I-') 


(XXIII)  (Poisson  (0)  has  pgf  exp{0(s-l)}) 

HBRMITE  has  pgf  expO^s-D+SjCs2-!)}, 

(XXIV)  GENERALIZED  HBRMITE  has  pgf  exnCe^s.lUD  fsm-n)  (m>2), 

(XXIV)'  GENERALIZED  Gj ,G2-HBRMITE  has  pgf  oxp{01 (Gj (») - 1) +0m (c;2 (sm) - 1) ) . 
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TABLIi  3 


UAMAGti  MODBL  CHAR ACTUIU2AT IONS 


If  Y  and  2  are  mutually  independent  and  distribution  of  Y,  given 


a+£)  is-. 

then 

distributions  of  Y  and  £  are; 

(XXV) 

Poi  s 

Hypergoomctrie 

Negative  Binomial 

I'olyu 

Hypergoometrie 

(XXVI) 

Multinomial 

Independent  Poisson 

Multivariate  llypergcomotric 

Multivariate  Negative  Binomial 

(XXVI J) 

Quasi -bi  nomiai  1 

Lagrange  double  Poisson 

Qua  s  i  -  hy  pe  rg  e  onio  t  r  i  c 

Quasi -binomial  I 

Quasi -Polya 

Qunsi -hypergeometri c 
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RESPONSE  FOR  A  GIVEN  STIMULUS 
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The  crew  compartment  of  an  army  vehicle  is  protected  by  an  armor 
plate.  It  is  desired  to  test  the  strangth  of  this  armor  plate  in  order 
to  assess  its  appropriateness  for  use  in  the  vehicle. 

A  Hpocimun  of  the  plate  is  taken  and  projoctiloH  aro  fired  nt 
different  points  on  the  plate  at  different  striking  velocities.  If  a 
projectile  pone trn ten  the  armor  It  in  said  to  have  defeated  the  armor. 
Our  goal  is  to  determine  the  relationship  between  the  striking  veiocity 
and  the  probability  of  penetration.  Due  to  the  exponnivo  nnturo  of  nil 
i  turns  involved,  this  goal  must  be  achieved  with  a  minimum  amount  of 
testing.  A  Bayesian  approach  for  solving  this  problem  ie  presented 
here  and  illustrated  using  some  real  date. 
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1.  STATEMENT  OF  THE  PROBLEM 

The  following  statement  of  the  problem  is  based  on  our  several 
discussions  with  Dr.  Robert  L.  Launer  of  the  Army  Research  Office,  Re¬ 
search  Triangle  Park,  North  Carolina,  and  Dr.  J.  Richard  Moore  of  the 
Ballistic  Research  Laboratory  (BRL),  Aberdeen  Proving  Ground,  Maryland. 

The  crew  compartment  of  an  army  vehicle  is  protected  by  a  certain 
kind  of  material  which  we  will  refer  to  as  an  "armor  plate."  It  is 
desired  to  test  the  strength  of  this  armor  plate  so  that  we  may  be  nble 
to  assess  its  appropriateness  for  use  on  the  vehicle. 

In  order  to  do  this,  a  10'  x  10'  specimen  of  the  armor  plate  is 
taken,  and  a  projectile  is  fired  from  a  gun  which  is  aimed  at  different 
points  on  the  plate,  In  Figure  1.1  below,  we  indicate  a  possible  firing 
pattern  according  to  which  the  gun  is  aimed. 

Typically,  the  distance  between  the  muzzle  of  the  gun  and  the 
target  in  about  200  meters,  and  the  velocity  of  the  projectile,  measured 
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Figure  1.1— Illustration  of  e  firing  pattern  of  e  gun. 

between  two  conveniently  located  points  between  the  gun  and  the  target 
la  about  5000  feet  per  second. 

The  projectile  la  known  aa  the  "panetrator,"  and  the  outcome  of 
each  firing  is  described  by  a  binary  variable  which  takes  the  value  1  if 
the  panetrator  defeats  the  target,  and  the  value  0  if  the  penetrator 
fails  to  defeat  the  target.  The  penetrator  Induces  a  stress  on  the  armor; 
the  stress  is  a  function  of  two  quantities,  the  "striking  velocity"  and 
the  "angle  of  fire,"  tfhe  striking  velocity,  also  known  as  the  "stimulus," 
is  the  velocity  with  which  the  penetrator  strikes  the  armor,  whereas 
the  angle  of  fire  9  (indicated  in  Figure  1.2  below)  is  the  amount  by 
which  the  armor  plate  is  tilted. 


Line  of  Fire 


Figure  1.2 — Illustration  of  the  angle  of  fire. 
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Both  the  armor  specimen  and  the  penetrator  are  very  expensive  and 


thus  the  testing  has  to  be  kept  to  a  bare  minimum.  One  strategy  that 
has  been  adopted  is  to  fix  the  angle  of  fire,  say  at  6*  ,  and  then  to 
fire  the  penetrator  at  different  striking  velocities.  After  each  firing, 
a  record  is  made  of  whether  the  penetrator  defeated  the  targot  or  not. 


It  la  assumed  that  the  striking  velocity  can  be  measured  without  any 
error. 


^  V^pqpfT^f  tpyi;;^  |  f.i;  i  p  ip  nj  *■-<■.  ■ ., 


2.  GOALS,  OBJECTIVES,  AND  SOME  COMMENTS 
ON  CURRENT  APPROACHES 

Given  that  our  goal  la  to  be  able  to  assess  the  appropriateness 
of  the  armor  plate  for  use  on  a  vehicle,  our  objective  should  be  to  es¬ 
timate  the  relationship  between  the  striking  velocity  (the  stimulus)  and 
the  probability  of  penetration  (a  response  of  1) .  This  is  illustrated 
in  Figure  2,1,  wherein  it  is  assumed  that  the  probability  of  penetration 
la  a  nondecreasing  function  of  the  stimulus. 

The  situation  described  above  is  identical  to  the  one  encountered 
in  "bioaesay  experiments,"  and  "low  dose  radiation  experiments,"  in  which 
the  relationship  mentioned  before  is  known  as  the  quantal  response  curve. 
The  dose  level  of  a  drug  la  the  stimulus,  and  Interest  generally  centers 
around  V  j  ,  the  stimulus  at  which  the  probability  of  response  is  .5  . 
Since  it  is  possible  to  subject  more  then  one  animal  to  a  particular 
does  level,  the  number  of  tests  at  each  value  of  the  stimulus  can  be 
more  then  one.  Furthermore,  tests  are  often  conducted  at  several  doso 
levels,  and  thus  the  large  sample  theory  which  typically  justifies  in¬ 
ference  from  bioaesay  experiments  is  adequately  substantiated. 


Probability  of 
Penetration 


Figure  2, 1— Probability  of  penetration  vs.  stimulus. 
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Illililfy.t'UJ  I  HI  In 


Despite  these  conspicuous  differences  between  bioassay  experi¬ 
mentation  and  the  problem  described  here,  the  methodology  and  techniques 
of  the  former  have  been  directly  adopted  for  use  In  the  latter.  In  so 
doing,  a  serious  compromise  has  been  made-- the  estimation  of  V  -  , 
rather  than  the  entire  quantal  response  curve,  has  been  made  the  domi¬ 
nant  issue  of  the  kinetic  energy  penetration  problem.  Specifically,  the 
BRL's  commonly  used  "Langley  Method"  [Rothman,  Alexander,  and  Zimmerman 
(1965,  pp.  55-58)]  and  the  "Up  and  Down  Method"  fop,  cit.,  pp.  101-103] 
focus  exclusive  attention  on  the  estimation  of  V  j  , 

The  typical  approach  used  in  bloassay  for  estimating  V  ^  is  to 
assume  that  the  probability  of  response  p  is  an  arbitrary  nondecreasing 
function  of  the  stimulus  V  ,  specified  via  the  relationship 

p  ■  F((v-y)/o)  , 

where  P  is  a  distribution  function  determined  by  a  symmetrical  density 
function  with  location  parameter  y  ‘and  scale  parameter  a  .  Often  F 
is  taken  to  be  the  normal  distribution  function 

x  ,  ,2 

F(x)  -  /  -i-  e"*8  ds  , 

-  ffi 

«,  V  mm  1 

or  the  logistic  distribution  function  F(x)  ■  (1  -  e  ) 

The  data  from  a  bioassay  experiment  consists  of  n^  ,  the  number 
of  subjects  receiving  stimulus  ,  1-1,..., K  ,  and  X^  ,  J-l,...,n^  , 
where 

“  1  ,  if  tho  J  subject  responds  under  stimulus  V^,  and 
-  0  ,  otherwise. 

Given  the  data  (n^,X^)  ,  ,  j-1 . n^  ,  the  param¬ 

eters  y  and  o  are  estimated  using  the  method  of  maximum  likelihood, 
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under  the  assumption  that  the  test  results  can  be  judged  independent. 

Once  y  and  o  are  estimated}  the  estimation  of  V  ,  follows  from 

•  »? 

the  fact  that  F  ,  the  tolerance  distribution,  has  been  specified.  Non- 
parametric  and  robust  estimators  of  V  ^  ,  auch  as  the  Spearman-Karber 
estimator,  the  L-astimator,  the  M- estimator,  and  the  Tukey  Biweight  es¬ 
timator,  have  also  been  obt lined,  all  under  the  aesumptlon  that  the 
density  function  giving  F  le  symmetric.  These  estimators  have  been 
discussed  by  Miller  and  Halpern  (1979),  Furthermore,  it  has  been  empir¬ 
ically  shown  that  for  the  estimation  of  V  .  it  does  not  matter  what 
specific  form  is  chosen  for  F  {  many  of  the  commonly  used  nonparametric 
estimators  yield  identical  estimates  of  V  ^  ,  as  long  as  symmetry  ie 
assumed. 

A  drawback  of  the  assumption  of  symmetry  is  that  the  estimate  of 
the  probability  of  response  whan  the  stimulus  is  zero  is  nonzero, 

Whereas  this  may  not  be  too  disturbing  in  bioassay  with  its  emphasis  on 
V.2  ,  in  the  problem  considered  here  and  the  low  dose  radiation  experi¬ 
mentation,  auch  an  estimate  would  be  clearly  unacceptable.  A  zero  value 
of  the  stimulus  should  correspond  to  a  zero  value  for  the  probability  of 
response. 

In  view  of  the  above  difficulty,  the  paucity  of  data  at  each 
level  of  the  stimulus,  and  our  inability  to  specify  a  functional  form 
of  F  which  has  Borne  practical  merit,  we  are  motivated  to  advocate  a 
Bayesian  approach  for  the  solution  of  this  problem,  Our  approach  is 
described  in  Section  3. 
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3.  AN  OUTLINE  OF  A  BAYESIAN  APPROACH 


A  Bayesian  approach  Co  Che  bioassay  problem  was  first  proposed  by 
Kraft  and  Van  Eeden  In  1964,  and  was  more  fully  developed  by  Ramsey  in 
1972.  We  consider  here  the  theme  proposed  by  Ramsey;  extensions  of 
this  theme  are  considered  by  Shaked  and  Singpurwalla  (1982). 

Let  0  =  Vq  <  <  . . .  <  VM  <  VM+1  <  »  ,  be  M  distinct  levels 

of  the  stimulus  at  which  the  target  (armor  plate)  is  tested;  M  is 
choson  in  advance.  The  outcome  of  a  test  at  1b  described  by  a 

binary  (0,1)  variable  X^  ,  where  X^  ■  1  if  the  penetrator  with  a 
striking  velocity  defeats  the  target.  Let  p^  ■  PfX^-l)  ,  1>1,...,M  , 

and  without  loss  of  generality,  we  assume  that 

O  3  <  <  Pj  <  ...  <  PM  <  Pm+1  5  1  !  (3.1) 


it  is  always  possible  to  choose  and  which  satisfy  the  above 

inequality. 

Given  X  *  (X^,...,^)  ,  one  goal  is  to  estimate  the  unknown 

pj'a  ,  i-1 . M  ,  subject  to  the  inequalities  (3.1).  Another  goal  is 

to  estimate  p.  ,  for  some  jj<i  ,  such  that  if  V.  <  V,  <  V.  ...  ,  the 
J  i  J  i+i 

estimates  satisfy  p^  <  <  p^^  ,  i*l . M  ;  this  pertains  to  esti¬ 

mating  the  probability  of  response  at  a  stimulus  where  no  target  was 
tested.  Yet  a  third  goal  would  be  to  estimate  the  largest  stimulus, 
say  Va  ,  for  which  pa  <  a  ,  where  0  <  a  <  1  is  specified. 

Ramsey's  approach  for  achieving  the  above  goals  is  to  assign  a 
Dirlchlet  as  a  prior  distribution  for  the  successive  differences 
Pi'  ^2”^1 . PM-PM-i  »  anc*  c^en  t0  uae  t*lc  moc*al  value  of  the 
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resulting  joint  posterior  distribution  as  a  Bayos  point  estimate  of 


(p^,,..,p^)  .  The  modal  value  is  computed  with  the  inequalities  (3.1) 
being  satisfied.  The  modal  value  of  the  posterior  distribution,  if 
unique,  is  also  known  as  the  generalized  maximum  likelihood  estimator 
(see  DeGroot  (1970,  p.  236)],  and  is  used  as  a  Bayes  estimator  when  we 
do  not  wish  to  specify  a  particular  loss  function.  Having  estimated 
the  p^'s  ,  the  estimation  of  p^  and  is  undertaken  via  an  inter¬ 

polation  procedure. 

Specifically,  if  >  0  ,  i«l,...,M  ,  and  B  >  0  are  constants 
such  that  *  1  ,  then  the  prior  density  function  it  is  of  the 

form 


TT  * 


M+l 

11  <Pi  "  Pi„i> 
U-l  1  1  1 


“i)s 


(3.2) 


It  is  important  to  note  that  when  averaging  according  to  TT  in¬ 
tegration  must  be  done  with  respect  to  dp^  /  (p^  -  P^_^)  > 

Since  M  has  been  prechosen,  the  stopping  rule  is  clearly  de¬ 
lineated,  and  so  the  likelihood  for  the  response  probabilities  at  the 
observed  stresses  in 


M  X,  1-X. 

n  vL  a  -  p.)  .  (3.3) 

i»i 

The  Joint  density  function  of  the  posterior  distribution  of 
Pi* ’"’Pm  *b  ProPort^ona^  t0  the  product  of  the  prior  density  function 
(3.2)  and  the  .likelihood  function  (3.3).  Thus 
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f(p1>....pM  I  \ . v 


M+l  X,  1-X, 

«  n  p/  a  -  PA)  1 

i»i  1  1 


m 


M+l 

n  r(ea.) 

.i-i  1  . 


i+l  a, )  8 

I/Pl'W  j  •  <3* 


M+l 

i 


4) 


Ramsey  has  not  bean  able  to  obtain  the  posterior  marginal  dis¬ 
tributions  of  p^  i  1-1,...,M  ,  nor  has  he  commented  on  any  aspects  of 
these  distributions.  He  uses  a  nonlinear  programming  algorithm  to  ob¬ 
tain  (Pi»>..,PM)  i  the  modal  value  of  (3.4),  subject  to  the  constraint 
that  <_  Pj  p^j  |  this  Is  his  Bayes  estimator  of  (plt...,PM)  . 

In  contrast  to  this  Maszuchi  (1982)  has  been  able  to  obtain  all  the 
moments  of  the  marginal  posterior  distribution  of  the  ,  1*1,... ,M  . 
This  work  of  Mazzuchi's  represents  an  extension  of  Ramsey's  results, 
and  is  one  that  takes  us  a  step  closer  to  a  fully  Bayesian  analysis. 

The  moments  can  be  used  to  approximate  the  marginal  posterior  dlstrl- 

'  I- 

buttons  of  the  p^s  using  the  techniques  given  in  Elderton  and  Johnson 
(1969).  The  approximated  posterior  distributions  give  us  a  measure  of 
uncertainty  associated  with  our  using  the  first  moment  of  the  marginal 
posterior  distribution  of  p^  ,  i-l,...,M  ,  as  our  Bayes  estimate  of 
p^  .  The  first  moment  of  the  marginal  posterior  distribution  la  used  as 
a  Bayes  estimator  when  we  are  willing  to  assume  the  square  error  as  a 
loss  function.  The  formulae  for  the  moments  and  their  use  for  approxi¬ 
mating  the  marginal  posterior  distributions  art  given  in  Appendix  A. 

The  computational  effort  required  to  compute  the  moments  men¬ 
tioned  above  increases  with  M  .  Thus  there  is  a  trade-off  between  the 
convenience  of  using  an  optimization  algorithm  to  obtain  the  modal  value 
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3.1  Specification  of  the  Prior  Parameter* 


In  order  to  implement  the  Bayesian  procedure,  we  need  to  specify 
the  prior  parameters  ,  1-1, . . . ,M  ,  and  3  ,  given  In  (3.2).  In  order 
to  do  thia,  we  observe  (see  Ramsey)  that  ui  ■  pA  -  p^  ,  i**l,,,,,M  , 
has  a  beta  distribution  on  the  unit  interval  (denoted  as 
u1  ~  Beta(Salt  0(1  -  o^)}  0,1))  , 


f(ul{  0a1,  0(1-^))  - 


0a,  0(l-a. ) 

u.  1  (1“U. )  1 


0  <  u,  <  1 

*»*  1  lb* 


E(u^)  ■  ,  and 

a* a  -  Oj) 


(3.5) 


var(Ui)  -  -^rrr  •  <3*6> 

If  PJ  denotes  our  best  prior  guess  about  ?i  ,  consistent  with 
the  fact  that  the  P£'s  increase  in  i  ,  then  the  a^'s  can  be  ob¬ 
tained  via  (3.5)  as 


J  ■'  J|* 


“l  ■  pi*  -  Fi-1  • 


1*2, , , .  ,M  , 


“M+l  “  1  "  PM  ‘ 

In  order  to  choose  the  parameter  $  ,  we  need  co  have  some  Idea 
about  the  uncertainty  associated  with  our  choice  of  p£  .  This  in 
practice  can  be  done  in  one  of  the  following  two  ways: 

(1)  Suppose  that  in  addition  to  ,  our  best  guess  about  the 
variance  of  p^  is  Var(p^)  .  Then,  substituting  *  pj 

in  (3.6),  we  have 

01,(1  -  a-) 

VarCu^  -  Var(Pl)  -  , 


so  that 


P$(l  -  Pj) 


-  -  1  ,  if  3  >  0  , 


VO  ,  otherwise. 

Note  that  0*0  corresponds  to  the  case  of  isotonic 
regression. 

(ii)  Often  in  practice  [cf.  McDonald  (1979)),  associated  with 

the  best  guess  value  p£  ,  a  user  is  able  to  specify  two 

l 

numbers  aj  >  0  and  b£  <  1  ,  such  that  for  some  y^ 
(specified  by  the  user),  0  <  y^  <  1  , 


P(a*  <  P:  <  b*) 


1  -  Y, 


Since  -  Deta(0ait  0(1-0^);  0,1)  ,  given  pj  ,  we  set 
^1  "  '  nn<*  that  valUG  of  0  such  that 
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b*  r,,v  Ba.-l  0(l-a. )-l 

TTso^T  rfea-^))  pi  <1"pi)  dp] 

1 


1-Yi 


(3.7) 

Suppose,  further,  that  for  any  one  or  more  of  the  indices 
1  ,  i*2,...,M  ,  a  user  is  also  able  to  specify  two  numbers 
*i  >  pt-l  *  *nd  hj  <  1  ,  such  that  for  some  y^  (speci¬ 
fied  by  the  user) ,  0  <  ^  <  1  , 

PUJ  <  (Pf  I  Pj.l . Pj)  <  b*)  -  1  -  Yt  . 

Then,  uaing  the  fact  (see  Ramsey)  that 

I  p}_1)  ~  Beta(Salt  3(1  -  -  ...  -  ct±) ;  pj^,  1) 

■  *(Pi  I  Pj_Li  M*)  *  say, 

we  can  find  the  smallest  value  of  0,0*,  which  satisfies 
(3.7)  and  (3.8),  where 


b* 


0  f<Pi  1  pi-lJ  e*ai)dpi  "  1  "  > 


with  a. 


(3.8) 


'i  ~  pi  ”  pi-l  »  • 

A  computer  code  which  determines  the  smallest  value  of  0  described 
above  is  available;  the  details  of  this  program  are  given  by  Mezzuchi 
and  8oyar  (1982).  Our  reason  for  choosing  the  smallest  value  of  0 
•tarns  from  the  fact  that  large  values  of  0  give  a  very  Btrong  prior, 
with  the  renult  that  even  a  large  amount  of  failure  data  will  not  change 
our  prior  distribution. 


39 


3 . 2  Interpolation  Procedure  and  the 
Estimation  of  Quantiles 


Let  the  M  dimensional  point 


(pl . PM? 


|(p^, • . . .p^)  ,  if  the  mode  of  the  joint  posterior  is 

(p.,...,pM)  ,  if  the  first  moments  of  the  marginal  poste¬ 
rior  are 


used  as  the  Baves  estimator  of  (p^,...,pM>  . 

Suppose  that  we  wish  to  estimate  p^  ,  for  some  Ji*  i  ,  1"1,...,M  , 
where  vt  <  vj  <  Vi+i  '  ^et  pj  *5e  our  best  prior  guess  of  ,  the 
probability  of  response  at  a  nonexperimentai  impulse  .  Then,  foliow- 

f 

ing  Ramsey,  we  pick  p^  in  such  a  manner  that 


Pi+1  *  Pj 


5*  -  D* 


+  +  ‘ 
Pj  ’  >i 


(3.9) 


For  the  estimation  of  Va  ,  the  ath  quantile  (0  <  3  <  1)  ,  we 

first  see  if  there  is  an  observation .stimulus,  say  ,  for  which 

p*  *  a  .  If  so,  then  is  our  Bayes  estimate  of  Va  .  If  not,  we 

determine  the  pair  of  observational  impulses,  say  and  *  for 

*4* 

which  <  a  <  Pi+1  •  Since  the  probability  of  response  rurve  is  as¬ 

sumed  to  be  increasing,  the  straight  line  segment  joining  the  pointB 
0,  p^,  p^,  p^,  •••»  PM>  *  wi^  b®  an  increasing  function  of  i  . 

We  shall  find  that  value  of  the  impulse,  say  V*  ,  <  V+  <  ,  for 

which  p*  ■  a  . 
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4. 


APPLICATION  TO  SOME  BRL  DATA 


In  Appendix  B  we  present  eight  sets  of  data  labelled  1,  2,  3,  4, 
6,  7,  8,  and  9,  pertaining  to  60  kinetic  energy  penetration  tests. 

These  data  were  given  to  us  by  Dr.  Moore  of  BRL  and  have  been  carefully 
sanitized  to  maintain  confidentiality.  Data  sets  labelled  5  and  10, 
also  given  to  us  by  Dr.  Moore,  have  been  eliminated  from  consideration 
because  the  striking  velocity  for  these  data  is  much  too  different  from 
those  of  the  other  sets.  All  the  10  sets  of  data  were  obtained  sequen¬ 
tially  over  time,  in  the  sense  that  data  Bet  1  was  the  first  one  to  be 
obtained,  followed  by  data  set  2  (obtained  after  some  lapse  of  time), 
and  so  on,  until  we  reach  data  set  9,  which  is  the  last  considered  here. 
To  the  best  of  our  knowledge,  all  eight  data  sets  are  assumed  to  have 
been  collected  under  identical  conditions.  That  is,  tlvre  is  no  indica¬ 
tion  that,  except  for  differences  in  striking  velocity,  the  material  and 
the  methods  of  testing  used  for  data  set  1  are  different  from  those  used 
in  data  set  2,  and  so  on.  This,  plus  the  sequential  nature  of  the  data, 
enables  ub  to  ubc  the  posterior  obtained  from  one  data  set  as  the  prior 
for  the  next  set,  and  go  on,  until  we  obtain  the  posterior  UBing  datu 
set  9,  which  then  gives  our  final  estimate  of  the  response  curve. 

Data  set  1  consists  of  13  observations  taken  at  striking  veloci¬ 
ties  ranging  from  128.60  (in  some  unspecified  units)  to  166,16.  The 
result  of  each  test  is  indicated  by  a  binary  variable  .  The  best 
prior  guess  values  ,  necessary  to  choose  the  prior  parameters  , 
were  not  specified  by  BRL.  However,  what  appears  to  be  reasonable  is  to 
assume  that  the  probability  of  response  at  a  striking  velocity  of  100  is 
close  to  zero,  and  that  at  a  striking  velocity  of  200  it  is  almost  1. 
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Thus  we  make  an  arbitrary  choice  for  p*  ,  say  pJq  ,  by  letting 
p*0  ■  1  -  exp[-.07(Vi  -  100)]  .  Data  on  striking  velocities  outside  the 
range  of  100  to  200  were  excluded.  Despite  this  arbitrary  choice  of 
p£q  ,  we  shall  see  how  even  a  scant  amount  of  data  significantly  changes 
the  posterior  response  curve,  provided  that  the  smoothing  parameter  $ 
la  not  too  large.  Three  values  of  S  were  also  chosen  arbitrarily; 
these  are  1,  10,  and  25.  Recall  that  small  values  of  $  tend  to  em¬ 
phasize  the  data,  whereas  large  values  of  0  tend  to  emphasize  the 
prior  distribution.  In  Appendix  B  we  show  our  analysis  for  the  case 
of  3  -  10  . 

Since,  in  reality,  the  data  are  generated  ^bquarttially  over  time, 
our  first  step  would  be  to  revise  the  best  prior  guess  valuea  pj0  , 
i-l . 61  ,  based  on  data  set  1  alone.  The  posterior  (modal)  valuea 

4) 

corresponding  to  the  striking  velocities  of  data  set  1,  p^  ,  will  be 
the  revised  values  of  p*Q  ,  for  i-1,.,,,13  ;  these  are  given  in  column 
5  of  the  table  in  Appendix  B.  The  revised  values  of  p*Q  ,  for  i"4,,..,61  , 

are  obtained  via  the  interpolation  formula  (3.9),  using  p^  ,  i**l . 13  , 

and  p*Q  ,  i-14,,.,,61  .  Let  the  revised  values  of  pj^  ,  i-*14,...,61  , 
be  denoted  by  p*^  ;  these  too  are  shown  in  column  5  of  the  table  in 
Appendix  B. 

Upon  receiving  data  set  2,  we  revise  the  values  p^  ,  i-14 . 19  , 

by  the  posterior  modal  values  corresponding  to  the  six  striking  veloci- 
ties  of  data  set  2.  We  denote  these  revised  values  by  p^  ,  I»l4,...,x9  ; 
these  are  given  in  column  5  of  the  table  in  Appendix  B.  The  revised 
values  of  p£^  ,  i-20,,,.,61  ,  are  obtained  by  interpolation,  using 


a.'.  L-l  /U*  »• 
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4.  4- 

Ptl  ,  i«l, ... ,13  ,  Pi2  ,  i-14 . 19  ,  and  p*^  ,  i«20,...,6l  ;  we 

denote  these  revised  values  by  p£2  >  i"20,.,.,61  ,  and  Bhow  them  in 
column  6 . 

We  continue  the  above  scheme  of  systematically  revising  the 
p^.'a  ,  either  via  the  posterior  modal  values  or  by  interpolation,  until 
we  incorporate  the  effect  of  all  eight  sets  of  data.  Data  set  9,  the 
last  one  considered  here,  consists  of  eight  observations  taken  at  start¬ 
ing  velocities  ranging  from  -  144,83  to  ■  198.94  .  The  pos¬ 
terior  modal  values  corresponding  to  the  striking  velocities  of  data  set 
9,  p^g  ,  i«54,.,,,61  ,  are  given  in  column  12 ;  the  interpolated  values 
p|7  required  to  obtain  the  Pi8's  are  given  in  column  11,  Since  the 
p^'s  Incorporate  the  results  of  the  previous  seven  sets  of  data,  we 
claim  that  the  final  posterior  modal  values  p^g  ,  i“54,..,,61  ,  are 
based  on  the  results  of  all  the  testing.  Had  we  ignored  the  sequential 
nature  of  the  data  and  computed  the  posterior  modal  values  by  uBing 
Bayes  Theorem  on  the  best  prior  guess  valueB  p*Q  ,  1*1,.,,, 61  ,  then 
the  posterior  modal  values  corresponding  to  V14  through  would  be 

4 

different  from  the  p^a  values,  i«14,.,.,61  ,  given  in  the  table.  This 
difference  is  due  to  the  interpolation  scheme  that  is  used  to  constantly 
revise  the  best  prior  guess  values,  when  we  consider  the  data  setB 
sequentially. 

4* 

A.  plot  of  p^g  versuB  ,  i*54,,.,,61  ,  represents  our  final 
estimate  of  the  quantal  response  curve.  Estimates  of  the  probabilities 
of  response  at  striking  velocities  different  from  ,  i*54,...,61  , 
can  be  obtained  using  the  interpolation  formula  (3.9).  When  wo  use 
the  interpolation  formula  to  obtain  an  estimate  of  ,  for  some 


43 


■Vi  h  U  ,5  .  l/.l  ilf.KIJtfe,  i  i 


t  !.«-i  rauitticv  in.  .Mwukausfiw,; ».  xcujjm.p- 


U.UIUU 


j *1 , ... t S3  ,  we  need  to  specify  a  value  p*  ,  the  best  prior  guess  value 
of  Pj  .  Suppose  that  the  index  j  appears  in  data  set  k  ,  for  some 
k  <  9  ;  then  for  pj  we  will  use  p^  ,  In  so  doing,  we  will  heve  in¬ 
corporated  the  effect  of  the  last  data  set,  data  set  9,  in  our  obtaining 
the  estimate  of  Pj  ,  and  thus  achieve  a  certain  amount  of  smoothness. 
Note  that  the  effect  of  the  data  sets  between  k  and  9  is  already  pres- 
ent  in  our  estimates  p^  ,  i-54,...,61  ,  and  these  are  used  in  our  in¬ 
terpolation  scheme.  For  example,  suppose  that  we  wish  to  astimate  tha 
probability  of  response  at  a  striking  velocity  of  158.52,,  This  striking 
velocity  occurs  in  data  set  2,  and  lies  batwaen  the  striking  velocities 
148.97  and  159.15  of  data  aet  9.  Tha  index  j  corresponding  to  the 
value  158.82  is  17.  To  use  (3.9),  wa  identify  p*+1  and  p*+1  as 
being  .70499  and  .53014,  respectively,  p£  and  p^  as  .62881  and 
.42386  (sea  data  set  9),  and  p£  as  .64436  (see  data  sat  2),  and  com- 

•f 

pute  p^  as  our  estimate  of  pj  . 

In  Figures  4.1,  4.2,  and  4,3,  we  show  plots  of  our  Bayes  estimate 
of  the  probability  of  response  at  the  eight  striking  velocities  of  date 


aet  9,  for  8  ■  1,  10,  and  25  ,  respectively.  Also  shown  are  the  90X 
probability  of  coverage  intervals  for  each  estimate.  These  intervals 
are  obtained  using  the  moments  of  the  posterior  distributions  of  p^  , 
iI>54,,..,61  ,  and  then  using  tha  techniques  of  Elder ton  and  Johnson 
(1969)  to  approximate  the  posterior  distributions— see  Appendix  A.  On 
each  of  these  figures  we  also  show  a  graph  of  our  best  guess  values 

p*0  >  i“l . 61  ;  these  enable  us  r.o  see  how  the  data  have  changed  our 

prior  estimates.  We  observe  that  the  90*  probability  of  coverage  inter¬ 
vals  tend  to  ba  small  in  the  middle  of  the  range  of  the  striking  velocitleii. 
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In  Figure  4. 4,  we  superimpose  the  plots  of  Figures  4.1,  4.2,  and  4.3, 
order  to  give  a  perspective  of  the  effect  of  0  In  our  computations. 
It  appears  that  our  Bayes  estimates  for  the  three  cases  of  0  -  1,  10 
and  25  tend  to  converge  toward  each  other;  this  is  to  be  expected, 
since  we  have  61  observations  with  which  ws  revise  our  prior 
probabilities. 
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Momenta  of  the  Marginal  Posterior  Distributions 

.  .  i  i  1  i 

The  momenta  of  the  posterior  distribution  of  ,  i-l,...,M  , 
have  been  obtained  by  Mazzuchi  (1982);  a  formula  for  obtaining  theae  is 
givan  below.  A  computer  code  which  facilitates  the  computation  of  the 
momenta  la  described  by  Mazzuchi  and  Soyer  (1982) . 

Let  ^  a  1  -  ,  ,1-1 . N  ,  B(a,b)  -  r(a)r<b)  /  T(a+b)  ,  and 

X, 
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These  moments  can  be  used  to  approximate  the  posterior  dlstribu“ 
tion  of  Pj^  ,  f(p^)  ,  i-l,...,M  .  In  order  to  do  thlB,  we  consider  a 
system  of  frequency  curves  described  by  Glderton  and  Johnson  (1969) 
which  are  based  on  the  transforms  of  a  standard  normal  variate  Z  . 

The  system  of  curves  which  is  appropriate  to  our  problem  is  that  referred 
to  aa  the  "bounded  system  of  curves,"  denoted  by  Glderton  and  Johnson 

(1969,  p,  123)  as  Sfi  ,  and  described  by 

Z  -  y  +  8  Ln[(p  -  e)  /  (e  +  \  -  p  )]  ,  e  <  p  <  e+X  , 


SI 
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where  y  ,  6  ,  X  ,  and  e  are  parameters  whose  values  are  determined 
by  the  first  four  moments  of  f(p^)  about  its  mean. 

Hill,  Hill,  and  Holder  (1976)  give  a  computer  code  which  deter- 
minea  y  ,  6  ,  X  ,  and  e  from  the  first  four  moments  of  f(p^)  about 
its  mean.  Since  it  was  assumed  that  p^  <  p^  <  p^  ,  we  estimate 
X  and  e  from  the  Bayesian  estimates  of  the  p^  j  y  and  <5  are  ob¬ 
tained  from  the  computer  code.  Having  obtained  these  parameters,  the 
distribution  f(p^)  is  obtained  from  Elderton  and  Johnson  (1969,  p. 
130)  as 

f  <Pi)  ‘  ife  ‘  Vfl  i  (7  +  41”(^))  ]  • 


e  <  p,  <  e  +  X  , 


where  N  in  our  case  is  1. 


In  order  to  obtain  the  approximate  (l-y)X  probability  of  coverage 
intervals  for  each  p^  ,  which  contain  its  Bayes  estimate  p^  (mode  or 
mean),  we  use  the  fact  that  since 

z  -  y  +  6£n [ <p±-e) / (c+X-p^) ]  ,  e  <  pA  <  e  +  X  , 

Pt  -  X  exp[(^j  +  l]  +  e  . 

Thus,  to  find  two  numbers,  a  and  b  ,  such  that 


P{p*-a  i  Pi  <  P^+b}  ■  1  -  5  , 


P  -din 


/  X - A  +  y  z  - A  +  y  -  1 

}  \  p*+b-e  J 


and  solve  for  a  and  b  by  netting 


-  5  , 
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and 
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whara  iB  the  (l-(6/2))th  percentile  of  a  atandard  normal  distri- 

butlon.  Taking  c  -  max<a,b)  ,  wa  form  our  interval 

Pr{p^  -  c<pi<p^  +  c}  >  1-6. 

Thaaa  intervale  may  not  be  symmetric  about  the  mean  or  modal  estimate. 
This  case  arises  whan  the  boundaries  of  the  probability  of  coverage 
interval  exceed  the  boundary  of  the  variable.  In  such  cases  the  variable 
boundary  is  used  as  the  boundary  of  the  probability  of  coverage  interval. 
The  probability  of  any  aymmetric  Interval  about  the  mean  or  modal  esti¬ 
mate  may  be  obtained  by  proceeding  iu  the  reverse  or  the  above  and 
evaluating  the  interval  for  the  standard  normal  variate. 
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APPENDIX  3 


In  the  table  below  we  give  veluee  of  the  atrlklng  velocity 
the  reaponee  ,  and  the  beet  prior  gueae  veluee  p*Q  ,  1-1, ... ,61  , 
for  the  eight  eete  of  date  daecrlbad  In  Section  4.  We  alao  ahow,  for 
6  *  10  ,  the  revlaed  value*  at  pJQ  ,  p^  ,  or  pj^  baaed  on  date  aet 
J  .  j-1,2,3,4,6,7,8,9  . 
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Abstract 

For  an  irreducible  Markov  chain  whose  time  parameter  is  discrete  and  whose 
state  space  is  a  countable  discrete  set,  criteria  for  recurrence  and  transience 
are  obtained  by  constructing  supermartingales.  These  constructions  are  exten¬ 
sions  of  Foster's  criteria  for  recurrence  and  transience  in  terms  of  inequali¬ 
ties;  and  they  are  similar  to  the  construction  of  Lyapunov  functions  in  dynami¬ 
cal  systems.  Examples  to  which  the  criteria  are  applied  include:  pairs  of 
queues  with  priorities,  pairs  of  queues  in  parallel,  two-dimensional  positive 
random  walks,  and  competition  processes. 
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1 .  Introduction 

Either  the  states  of  an  irreducible  Markov  chain  are  recurrent  or  the/  are 
transient.  To  determine  which  occurs  is  the  first  step  in  analysing  the  chain. 
For  recurrent  chains,  the  next  step  is  to  determine  the  frequency/ of  visits 
of  the  sample  paths  to  each  state.  For  transient  chains,  the  next  step  is  to 
determine  the  asymptotic  behavior  of  the  sample  paths.  This  paper  deals  with 
criteria  to  distinguish  between  recurrence  and  transience  which  are  obtained 
by  the  construction  of  non-negative  supermartingales.  This  method  is  similar 
to  the  construction  of  Lyapunov  functions  to  analyze  the  stability  of  dynamical 
systems. 

Section  Two  contains  the  statements  of  the  criteria  and  their  proofs.  The 
results  presented  there  are  extensions  of  the  criteria  obtained  by  Foster  (1951, 
1952,  1953)  for  recurrence  and  transience  in  terms  of  inequalities.  Recently 
Mertens,  Samuel -Cahn,  and  Zamir  (1977,  1978)  have  also  used  supermartingales  to 
obtain  necessary  and  sufficient  conditions  for  recurrence  and  transience  of 
Markov  chains . 

Section  Three  deals  with  specific  examples:  pairs  of  queues  with  priorities, 
pairs  of  queues  in  parallel,  two-dimensional  positive  random  walks,  and  competi¬ 
tion  processes.  Criteria  for  positive  recurrence  and  transience  have  been 
obtained  previously  for  these  examples:  for  priority  queues,  by  Kesten  and 
Runnenburg  (1957);  for  parallel  queues,  by  Kingman  (1961b);  for  two-dimensional 
positive  random  walks,  by  Kingman  (1961a)  and  Malyiev  (1972);  and  for  competi¬ 
tion  processes,  by  Iglehart  (1964)  and  Reuter  (1961).  However,  the  proofs  pre¬ 
sented  here  are  different  from  those  in  the  papers  cited  above. 
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2 .  Criteria  for  Recurrence  and  Transience 


Let  (Zn:  n  j>  denote  a  stationary  Markov  chain  whose  tine  paramoter  is 
discrete  and  whose  state  space  is  a  countable  discrete  set,  S.  Throughout  this 
paper,  the  underlying  probability  space  (ft,F)  is  fixed  with  ft  the  sot  of  se¬ 
quences  of  elements  of  S  and  with  P  the  sigma-field  generated  by  the  finite¬ 
dimensional  cylinder  sets.  Then  the  random  variables  {Zn:  n  >  0}  are  simply 
the  coordinate  functions  on  the  product  space  ft.  The  chain  is  governed  by  a 
transition  function  P(x,x')  defined  for  elements  z,  s'  of  S.  Such  a  transition 
function  determines  a  family  of  probability  measures  { P  :  z  eS)  on  the  measure 
space  (ft,F).  Namely,  to  a  cylinder  set  of  the  form  {Zi  ■  Xj),  the  measure 
Px  assigns  the  value  6(jZ0,»)nj£j  PC**,  aitl).  (Here,  the  value  6(zfl,z)  is 
oqual  to  one  if  z»z0  and  to  zero  otherwise).  Finally,  it  is  assumed  that  tho 
chain  is  irreducible.  Then,  given  two  points  s  and  z *  of  S,  there  are  a  posi¬ 
tive  integer  k  and  points  ij,  z2, . . . of  S  such  that  the  product 
P(z,Zj){lljj£jp(Zj,  is  positive. 

Eithor  tho  states  of  an  irroduciblo  Markov  chain  aro  rocurront  or  they  uru 
transient.  The  distinction  is  that  the  typical  sample  path  of  a  recurrent 
chain  visits  every  state  infinitely  often  while  the  typical  path  of  a  transient 
chain  does  not  visit  any  state  infinitely  often.  Formally,  the  chain  is  recur¬ 
rent  if  there  exist  a  point  t*  of  S  and  a  finite  subset  K'  of  S  such  that  the 
probability  Pj/^cA*  infinitely  often)  is  positive.  Moreover,  if  this  occurs 

then  for  any  point  x  and  any  finite  subset  A  of  S,  the  probability  P„{Z_eA  i.o.) 

z  n 

is  equal  to  one.  The  chain  is  transient  if  there  exist  a  point  i*  of  S  and  a 
finite  subset  A'  of  S  for  which  the  probability  Pf*{ZneA'i.o.}  is  equal  to  zero. 
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For  a  subset  A  of  S,  the  stopping  time  is  the  non-negative  random  vari¬ 
able  defined  by: 

Ta  ■  min  (n>l:  ZneA^  if  the  set  is  not  empty; 

■  +  "  otherwise. 

For  recurrent  chains,  the  probability  PZ(TA  <  «•}  is  equal  to  one  for  any  point 

z  of  S  and  any  subset  A  of  S.  A  recurrent  chain  is  positive  recurrent  if  there 

exist  a  point  z"  of  S  and  a  finite  subset  A'  of  S  -  { z "}  for  which  is 

finite.  (Here,  E.  refers  to  the  integral  with  respect  to  the  measure  P  .) 

z  z 

Positive  recurrent  chains  are  exactly  those  which  admit  an  invariant  probability 
measure.  See  Chung  (196?)  or  Freedman  (1971)  for  a  more  complete  discussion  of 
these  results, 

To  show  that  an  irreducible  chain  is  transient  it  is  enough  to  show  that 
its  sample  paths  cannot  visit  certain  states  infinitely  often.  To  show  that 
an  irreducible  chain  is  recurrent  it  is  enough  to  show  that  paths  whose  initial 
point  is  outside  a  fixed  finite  set  are  certain  to  hit  that  set.  These  are  the 
basic  ideas  of  the  results  which  follow.  Their  proofs  make  use  of  the  notion 
of  a  superman ingale  and  of  the  convergence  theorem  for  non-negative  supermartin¬ 
gales.  The  necessary  material  is  summarized  in  Appendix  A. 

Theorem  2.1,  Let  (zn:  n^O?  denote  a  stationary  Markov  chain  taking  values  in 
countable  discrete  set  S.  Suppose  that  there  exists  a  non-negative  function  (j>, 
defined  on  S,  for  which 

(2.2)  E  M>CZn+1)|Zn  ■  z)  <  $(z)  for  all  z  S.  If  is  not  constant  and  if 
all  states  of  the  chain  communicate,  then  the  chain  is  transient. 

Proof.  Let  z  and  z“  be  two  points  of  S  for  which  $(z)  +  ^(z'').  The  sequence 
(<J>(Zn):  n><J}  is  a  non-negative  supermartingale;  it  converges  almost  everywhere 


62 


with  respect  to  the  measure  P  .  Since  all  states  communicate,  paths  of  a  recur* 

z 

rent  chain  would  visit  infinitely  often  each  of  the  points  z  and  2'.  But  this 
contradicts  the  convergence  of  the  sequence  ($(Zn):  n>0} . 

Theorem  2.3.  Let  { Zn :  n»A}  denote  a  stationary  Markov  chain  whose  state  space 
is  the  countable  discrete  set  S.  Suppose  that  there  exists  a  sequence  {$n:  n>0} 
of  non-negative  functions  on  S  for  which 

(2.4)  E(<KZn+j)  ] Zfl  ■  2)  <  for  all  xeS.  If  all  states  of  the  chain 

communicate,  and  if  there  is  a  point  xcS  for  which 

(2.5)  lim  n<wo  ♦„(*)  ■  +  *,  then  the  chain  is  transient. 

Proof .  Let  z  be  a  stste  of  S  for  which  the  property  (2.5)  holds.  The  non¬ 
negative  supermartingale  {$n(fcft)i  n>0)  converges  almost  everywhere  to  a  finite 

limit  with  respect  to  the  measure  P  .  Sample  paths  of  a  recurrent  chain  would 

z 

return  infinitely  often  to  the  initial  state  z;  but  then  (2.5)  contradicts  the 
result  of  the  convergence  theorem. 

Mertens,  Samuel-Cahn,  and  Zamir  (1977)  have  obtained  independently  the  next 
result  with  a  similar  proof. 

Theorem  2.6.  Let  { :  n>0}  denote  a  stationary,  irreducible  Markov  chain  tak¬ 
ing  values  in  a  countable,  discrete  set  S.  If  there  exists  a  finite  subset 
A  of  S  and  a  non-negative  function  defined  on  S,  for  which 

(2.7)  B(KZn+l)|Zn  *  z)  <  <Kt)  for  1  i  A,  and 

(2.8)  Us  <j>(z)<M)  is  a  finite  set  for  all  M>0, 
then  the  chain  is  recurrent. 

Proof.  Let  m  -  inf  (<Hz) :  zeS) ,  PTom  (2.8),  it  follows  that  there  are  states 
Zj  and  z2  of  S  for  which  <K*j)  ■  m,  ♦(z2)><Kzj)  ,  and  also  P(Zn+1  ■  i2\\  *  * j } >9 . 
Since  the  inequality  (2.7)  cannot  hold  for  Zj,  the  exceptional  set  A  is  not  empty. 


Let  icS-A.  To  show  that  the  chain  is  recurrent,  it  is  sufficient  to  show 
that  its  sample  paths  hit  A  almost  surely  with  respect  to  the  probability  mea¬ 


sure  P  .  The  non-negative  supermartingale  ($(2(nATA))}  converges  almost  every¬ 
where.  Because  the  chain  is  irreducible,  it  follows  from  C2.fi)  that 
Pj.flim  sup  n>+fl0  4>(Zn)  <»}■«.  Thus,  convergence  of  the  sequence  {$(2(nATA))J 
implies  that  paths  from  z  hit  A  almost  surely. 


The  remaining  results  of  this  section  refer  specifically  to  irreducible 
chains  { 2^ :  n>o}  on  the  integral  lattice  zjj}  in  of  points  whose  coordinates 
are  non-negative  integers.  For  any  vector  W  ■  (Wj,  W2...,W^)  e  R*\  the  "ex- 
ponential"  W  n  denotes  the  product 


Proposition  2.9.  If  there  exists  a  vector  W  •  (W^  W2,.,.,Wd)  whose  components 
are  strictly  positive  and  which  satisfies 

z  -  z 

(2.10)  E(W  *  °|Z0  ■  z)  <  1  for  all  z  c  /  ,  then  the  chain  is  tran¬ 

sient. 

Proof.  The  assertion  follows  from  Theorem  2.1  with  the  function  <j>(Z)  ■  WZ. 
Proposition  2.11.  Suppose  that  there  exists  a  vector  We  ^  whose  components 
satisfy  the  inequality  >  1  for  1  <  i  <  d  and  a  finite  subset  A  of  Z^  such 
that,  for  all  n>0, 

Z  -  Z 

(2.12)  E(W  n+*  n|zn  ■  z)  <  1  for  zcA,  then  the  chain  is  recurrent. 
Proof.  Note  that  the  origin  belongs  to  the  exceptional  set  A  and  apply  Theorem 
2.6  with  the  function  <p (Z)  ■  WZ. 

Proposition  2.13.  Suppose  that  there  exists  a  vector  W^c  whose  components 

satisfy  >  1  for  1  <  i  <  d  and  a  finite  subset  A  of  z^  such  that  K<  1  where 


2.-2 

(2.14)  K  *  sup  {  E(W  n+  n | ■  s):  n>0,  z  e 2^  -  A).  Then  the  chain 
is  positive  recurrent. 

Proof.  Note  that  the  exceptional  set  contains  the  origin.  Let  z  be  a  point  of 
Z^  -  A.  For  n>0,  it  follows  from  (2.14)  that 

V*2"41  •  >(TA>n.l)>  i  ■  1(TAi„.l))IV) 

'  Ez<w  "  '  l(TA  >  n)CE(WZn*1  '  S|z„» 

i  ltEzt"Zn  ■  ‘(Ta>„)>- 

z 

So  for  n>0,  E(W  n  .  1(T  >  nJ)  <  KnWz.  Finally,  since  W£  >  1  for  1  <i  <  d: 

A 

Z 

P2(Ta  >  n)  <  Ez(1(T  >  n)W  n)  <  K V. 


Therefore,  P2(TA  >  n)  converges  to  tero  geometrically.  Hence  EZ(TA)  is  finite 
for  each  positive  integer  k.  This  shows  that  the  chain  is  positive  recurrent. 

Note  that  the  construction  of  "exponential"  supermartingales  of  the  type 
described  in  (2.13)  provides  an  estimate  of  the  distribution  of  the  hitting 
time  T^  in  terms  of  a  geometric  distribution.  When  the  explicit  forms  of  the 
iterates  of  the  transition  function  are  not  simple  to  obtain,  such  estimates 
are  useful  for  approximating  the  moments  of  hitting  tiroes.  This  technique  has 
been  used  by  Kemperman  (1961).  A  related  technique  for  continuous -parameter 
chains  has  been  proposed  by  Aldous  (1981). 
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3.  Examples 

Example  A:  Pairs  of  Queues  with  Priorities.  Priority  customers  and  non¬ 
priority  customers  queue  up  at  a  counter  for  service.  Non-priority  customers 
are  served  only  when  there  are  no  priority  customers  in  the  system.  The 
arrival  rates  are  A j ,  for  priority  customers,  and  X2,  for  non-priority  customers. 
Their  service  rates  are  and  p2  respectively.  Intor-arrival  times  and  ser¬ 
vice  times  are  exponentially  distributed. 

Let  ■  (Xn»Y^)  represent  the  number  of  priority  customers  and  the  number 
of  non-priority  customers  in  the  system,  observed  just  after  the  n-th  change  of 
state.  The  transition  function  for  this  discrete  parameter  chain  is-,  if  x>0: 

P{Z1  -  (x+l,y)|zfl  ■  (x,y)}  -  Xx/r, 

PiZl  -  (x-l,y)|Z0  -  (x,y)}  -  ^/T, 

P(Z1  -  (x,y+l) | ZQ  -  (x,y))  -  X2/r. 

The  normalizing  constant  T  is  equal  to  ^^^l’  x"°  Rnd  y>0: 

P{Zj  -  OP,y*l)|z0  -  (0,y)J  -  X2/r«, 

P(Zj  -  (e,y-l)|z0  -  (o,y)}  -  V2/T", 

PC2x  -  Cl.y) |ZQ  -  (0,y)}  -  Aj/P". 

The  normalizing  constant  T"  is  \  +  *2+^2<  For  jumps  from  (0,0): 

P(Z1  -  (1,0)  |Zfl  -  (C,0)>  -  yCAj+Aj). 

P(21  -  (fi,l)|Z&  -  (0,0)}  -  x2/(A1+x2). 

Criteria  for  recurrence  and  transience  can  be  described  simply  in  terms  of 
the  parameters  Aj,  A2»  Wj*  an^  W2* 

Theorem  3.1.  The  queue  with  priorities  is  transient  if: 

(3.2)  A j  >  y j , 


66 
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or  if  both 

(3.3)  Xj  <  ux  and  X1/y1  +  X2/u2  >  1, 

or  if 

(3.4)  Xj  -  vv 

It  is  positive  recurrent  if 

(3.5)  X1/y1  ♦  X2/y2  <  l. 

It  is  recurrent  if 

(3.6)  Xj  <  )Jj  and  ^j/Pj  +  ^2^2  *  ^ ‘ 

The  following  notation  will  be  used  in  the  proof.  Let  M  ■  E(zn+J2n  •  (x,y)) 
for  x>0;  so  ■  (Xj-pijJ/r  and  Mj  ■  X2/r.  Let  M"  ■  E(zn+  ^  1 2n  ■  (0, y) )  for  y>0; 
then  M"  »  Xj/r*'  and  M!£  ■  (X2~y2)/r",.  Finally,  let  M°  ■  E(Zn+1 1 2n  ■  (0,0));  so 
■  X1/(X1+X2)  and  m|  ■  Xj/fX^+Xj).  These  vectors  represent  the  mean  displacement 
due  to  a  single  step  of  the  chain.  Note  that  the  components  M°,  M2,  M2,  and  M'j' 
are  positive. 

Proof  of  Theorem  3.1.  Given  a  vector  W  ■  (W^,W2),  set  V  ■  (W^-l,  W2>1).  If 
|vj  <  1,  then  Wj1  -  1  -  *  W^VjJ.  Thus 

z  •  z 

(3.7)  E(W  n+1  n  |zn  -  (x,y))  -  1  +  (M,V)  +  (Wj/nvjJwJ1  if  x>0; 

■  1  +  (M",V)  +  (Pj/OVjWj1  if  x*0,  y>0; 

■  1  +  (M°,V)  if  x»y-0. 

If  it  is  possible  to  choose  the  vector  V  so  that  each  of  the  inner  products  (M,V) 
(M",V),  and  (M°,V)  are  strictly  negative,  then  by  choosing  ||v|i  sufficiently 
small,  it  is  possible  to  have  the  entir«  right  side  of  equation  (3.7)  strictly 

negative, 

Now,  if  (3.2)  holds,  apply  (2.9)  with  W  »  (1-V^,1)  for  VjE(0,l)  sufficiently 
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small  to  conclude  that  the  chain  is  transient. 

If  either  (3.3)  or  (3.4)  holds,  it  is  possible  to  choose  Vj  and  V2  from 
the  interval  (-1,0)  so  that  (M,V)  <  0  and  (M",V)  <  0.  For  such  a  vector  V, 
automatically  (H°,V)  <  0,  so  again  it  follows  from  (2.9)  that  the  chain  is 
transient . 

If  (3.5)  holds,  it  is  possible  to  choose  Vj  and  V2  from  the  interval  (0,1) 
so  that  (M,V)  <  0  and  (M",V)  <  0.  With  j ) V | j  small  enough  and  the  exceptional 
set  A  *  {(0,0)},  it  follows  from  (2.l3)that  the  chain  is  positive  recurrent. 

If  (3.6)  holds,  then  the  vectors  M  and  M"  point  in  opposite  directions. 

Let  U  -  (M2,-Mj).  This  vector  has  positive  components  so  the  random  variables 
4>(Zn)  ■  (Zn,U)  are  non-negative.  With  A  -  {(0,0)}  as  the  exceptional  set,  it 
follows  from  (2.6)  that  the  chain  is  recurrent. 

Example  B;  Pairs  of  Queues  in  Parallel.  Consider  a  counter  where  two  servers 
wait  on  arriving  customers.  The  servers  work  independently  but  at  equal  rates. 

An  arriving  customer  joins  the  shorter  of  the  two  lines;  if  both  lines  have  the 

same  length,  he  is  equally  likely  to  join  either.  It  is  assumed  that  the  inter¬ 

arrival  times  have  the  exponential  distribution  with  rate  A  and  that  the  service 
times  have  the  exponential  distribution  with  rate  U. 

Let  Zn  ■  ( Xn , Yn)  denote  the  number  of  customers  to  be  handled  by  the  servers, 
just  after  the  n-th  change  of  state.  The  transition  function  of  this  discrete¬ 
time  chain  is  shown  in  Figure  3.1.  There  it  is  assumed  that  the  time  scale  has 

been  fixed  so  that  A  ♦  2U  ■  1. 

Theorem  3.3.  The  discrete  parameter  chain  associated  with  the  pair  of  queues  in 
parallel  is  transient  if  A  >  2U.  It  is  positive  recurrent  if  A  <2U;  and  it  is 
recurrent  if  X  ■*  2p. 
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Let  M  *  E(2  -  Z  )  \Z)  and  note  that 

n  n+j  n  1  n 


(3.9)  M  - 
n 

(A/2  -p, A/2 

-y) 

if 

V 

Yn 

and  X_  >  0; 

n 

■ 

C-y.A-y) 

if 

V 

Yn 

*«•» 

O 

A 

- 

(A-p,-p) 

if 

V 

Xn 

>  0; 

3 

(1/ (A+p)) (*p 

.A) 

if 

V 

0  i 

and  Xn  >  0; 

■ 

(l/(A+p)) (A, 

-y) 

if 

Xn  * 

•  0 

and  Yn  >  0; 

■  • 

(1/2, 1/2) 

if 

xn  1 

-  yn  -  0. 

of  Theorem 

3.8.  If  A 

>  2y. 

then 

the 

inner  product  of 

direction  of  (-1,-1)  with  each  of  the  vectors  displayed  in  (3.9)  is  negative. 
Let  VjeCO.l)  and  set  V  ■  (-Vj.-Vj)  and  W  «  (l-Vj.l-Vj) .  For  n  >  0, 


E(W 


Zn+1  "  Zn, 


V  "  1  *  <Mn'V>  +  Vl®n' 


where  {©n:  n  >  0}  is  a  sequence  of  uniformly  bounded  random  variables.  Thus, 
if  Vx  is  sufficiently  small,  then  almost  surely: 


Z  -  Z 

E(W  n+1  n|Zn)  <1  fcr  n  >  0. 


That  the  chain  is  transient  follows  from  (2.1). 

To  conclude  that  the  chain  is  positive  recurrent,  apply  Foster’s  criterion 

with  the  function  <J>(z)  ■  x  ♦  y  and  the  exceptional  set  A  *  {(0,0)},  if  A  <  p. 

2  2 

If  p  <  A  <  2p,  the  function  #{z)  »  x  ♦  xy  +  y  will  do. 

In  the  critical  case,  consider  the  sequence  of  random  variables  (X*,  Y*) 
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defined  by  X*  ■  min(Xn,Yn)  and  Y*  ■  max(Xn,Yn).  (See  Figure  3.2  for  the  tran¬ 
sition  diagram  of  the  chain  (X*,Y*).)  It  follows  from  (2.6)  with  the  exceptional 
set  {(0,0)}  and  the  function  $(z)  ■  2x  ♦  y  that  the  chain  {(X*,Y*):  n>p)  is 
recurrent.  Since  (X*,Y*)  ■  (0,0)  if  and  only  if  (Xn,Yn)  ■  (0,0),  it  follows 
that  the  original  pair  of  queues  is  also  recurrent. 

Example  Ci  Two-Dimensional  Positive  Random  Walks.  Recall  that  l 2  is  the 
2 

lattice  in  ft  of  points  whose  coordinates  are  non-negative  integers.  This 
lattice  can  be  decomposed  into  four  types  of  states:  interior  points,  for  which 
both  coordinates  are  positive;  boundary  points  whose  first  coordinate  is  positive, 
boundary  points  whose  second  coordinate  is  positive;  and  the  origin.  The  transi¬ 
tion  function  for  a  two-dimensional  positive  random  walk  is  homcgenenous  with  re¬ 
spect  to  these  four  types  of  states,  and  single  steps  of  the  chain  lead  only  to 

2 

neighboring  points  in  Z+.  Let 


Pij  ■  p{Vi 

Pij  •  p<z„.l 

pij  •  PU 


h*l 


pij  ■  P(Z, 


n+1 


(x+i,y+j)|Zn  -  (x,y)>  for  x  >  0,  y  >  0;  |i|  <  1,  |j|  <  1; 

(x+i,  j)  |Zn  ■  (x,0)}  for  x  >  0,  |  i|  <  1.  0  <  ,1  £  1; 

(i.  y*j)  |Zn  *  (0,y)}  for  y  >  0,  |j|  <  1,  0  <  i  <  1; 

(ip  j)  |Zn  *  (0.  0)}  for  0  <  i.j  <  1. 


Criteria  for  recurrence  and  transience  can  be  stated  in  terms  of  the  drift 
vectors  due  to  single  steps  of  the  chain.  Let  M  ■  B -  Zn | Zn  ■  (x,y))  for 
x  >  0  and  y  >  0;  and  let  M'  ■  E(Zn+1  -  Zn | Zn  ■  (x,0))  for  x  >  0,  M"  - 

B*Zn*l  “  2Jzn  “  Sot  *  >0‘  Then: 

Theorem  3.10.  When  <  0  and  M2  <  0,  the  two-dimensional  positive  random  walk 

is  positive  recurrent  if  both  of  the  determinants 
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Mj  M|  M2  Mg 

Mj  Mj  M^' 

are  strictly  negative.  It  is  recurrent  if  at  least  one  of  the  determinants  is 
strictly  negative. 

Proof.  It  is  assumed  that  sufficiently  many  of  the  terms  of  the  distributions 
p,  P'«  P"f  and  p°  are  positive  that  the  chain  is  irreducible. 

The  criteria  for  transience  follow  from  Theorem  3  of  Kesten  (1971). 

Consider  the  function  $(»)  ■  ax2  ♦  2bxy  *  cy2.  Note  that  if  b2  <_  ac,  and 
if  a  >_  0  and  c  >_  0,  then  $  takes  only  non-negative  values.  For  n  >_  0: 

st*'WlV"  *<V  *  ((2‘\  *  2bV  2b*n  *  2oV-  Hn>  * 

■WVrV!  *  Sb'xn.l  -VtVl-V  *  •tW'W’lV' 

Since  a  single  jump  from  Zn  leads  only  to  neighboring  lattice  points,  it  follows 
that : 

EUCZn*l)|2n}  *  *  a  *  2lbl  +  c  +  ((2ftXn  +  2bV  2bXn  *  icYn},V' 

Moreovor: 

((2aXn  +  2bYn,  2bXn  ♦  2cYn),  Mn)  ■  2Xn((a,b) , (M>,M£))  if  Yfl-0,  Xn>0; 

■  2Yn((b,c),(M'1',M!j'))  if  Yn>0,  Xn-0; 

-  2Xn((a,b),(M1,M2))  +  2Yn((b,c),£Ml,M2)) 


if  X  *  Y  >  0. 
n  n 


Thus,  if  the  numbers  a,b,  and  c  can  be  chosen  so  that  a  >0,  c  >0,  and  b*<  ac; 
and  both 


(3.11)  ( (a,b) ,  (MrM2))  <  0  and  ((a.b),  (MJ,M$))  <  0; 

as  well  as 


(3.12)  ((b,c) ,  (MrM2))  <  0  and  ((b,c),  (My.Mjj'))  <  0} 

2 

then  there  is  a  finite  subset  A  of  Z+  such  that 

sup  E{<f>(Zn+1)  |zn  «  Z>  <  <Kz)  -  !• 
ztfA 

In  this  case,  that  the  chain  is  positive  recurrent  follows  from  Foster's  theorem. 
Given  that  both  determinants  are  strictly  negative,  there  is  a  vector  (a,b)  with 
a  >  0  and  b  <0  for  which  (3.11)  holds  and  a  vector  (b,c)  with  b  the  number  al' 
ready  chosen  and  c  >0  such  that  (3.12)  holds.  Since  ((a,b),  (M^,M2))  <  0  and 
((b,c),  CMX ,M2))  <  0,  it  follows  that 

-a I Mj |  +  | b [ | M2 |  <0  and  |b||Mj|-  c|m2J  <  0; 

Thus  |b|/a  <  IMj/mJ  and  |b|/c  <  Im^mJ  Finally,  b2  <  ac;  and  this  com¬ 
pletes  the  proof. 

Example  D:  Birth  and  Death  Processes.  Let  {  N(t):  t> 0}  denote  a  Markov  chain 
with  continuous  time  parameter  whose  state  space  is  Z2.  For  the  two  examples  con- 


sidered  here,  with  probability  one,  the  sample  paths  of  the  chain  have  only 
finitely  many  jumps  in  a  finite  time  interval.  If  N(t)  ■  z,  then  a  single 
jump  leads  only  to  one  of  the  neighboring  points  in  Z+  of  the  form  i  ♦ 

(Here  e^  denotes  the  unit  vector  for  which  the  i-th  coordinate  is  equal  to  one 
and  all  of  the  other  coordinates  are  equal  to  zero.)  Kesten  (1976)  has  given 
criteria  for  the  recurrence  and  transience  of  two-dimensional  birth-and-death 
processes  with  linear  transition  rates.  Milch  (1968)  used  generating  functions 
of  several  variables  to  study  special  examples  of  birth-and-death  processes  with 
linear  transition  rates.  Other  examples  of  multi-dimensional  birth-and-death 
processes  have  appeared  in  the  literature  with  the  name  "competition  processes." 
Such  processes  have  been  studied  by  Iglehart  (1964)  and  by  Reuter  (1961). 

In  the  following  examples,  the  results  of  Section  Two  lead  to  simple  proofs 
of  criteria  for  recurrence  and  transience. 

Suppose  that  the  transition  rates  are  given  by; 

P{N(t+h)  ■  (x+l.y)  |N(t)  ■  (x.y)>  ■  ah  +  o(h); 

P(N(t*h)  -  (x.y+1)  |N(.t)  ■  (x.y)}  -  &h  ♦  o(h); 

P{N(.t+h)  ■  (x-l.y)  |N(t)  ■  (x.y))  ■  yxh  ♦  o(h); 

P{N(t+h)  ■  (x,y-lj  |N(t)  ■  (x.y))  ■  dyh  +  o(h). 

This  example  was  considered  by  Reuter  (,1961).  If  the  pair  (x.y)  of  non-negative 
integers  represents  the  sizes  of  twc  populations,  then  these  transition  probabili¬ 
ties  correspond  to  a  process  in  which  the  populations  grow  independently. 

Suppose  that  the  numbers  a.p.y,  and  d  are  strictly  positive.  Then  all 

states  of  the  chain  communicate.  Let  (Z  :  n>0)  be  the  embedded  Markov  chain  of 

n  — 

the  successive  states  of  the  continuous  time  process.  Its  transition  function  is 


displayed  in  Figure  3.3.  Note  that  the  normalizing  constant  r(x,y)  is  defined 
by  r(x,y)  ■  CH'fi+Yx-'-fiy. 

Proposition  3.13.  This  embedded  Markov  chain  is  positive  recurrent. 

Proof.  Let  V  b(Vj,V2)  be  a  vector  of  R2  for  which  Vj  >  0  and  V2  >  0.  Set 
W  ■  (l*Vj,l*V.)  and  note  that: 

2  2 

B(W  ’  n|zn  -  (x,y))  -  1  +  (a-yxJV^r  ♦  (B-6y)V2/r+  Kx>yV2  ♦  K'  yV2 
where  K  ■  w'S'x/r  <1  and  K»  ■  W^dy/r  <  1. 

Xp/  i  -  *1/  *  “ 

Thus 

Z  ,  -  Z 

E(W  n+1  n|Zn  -  (x,y))  <  1  e  (a-YxJV^r*  (S-dy)V2/r  ■*■  { |v|  |a. 

If  ||vf|  is  sufficiently  small,  there  is  a  finite  subset  A  of  Z2,  necess- 
arily  containing  (0,0),  for  which 

2  *  "  Zn 

sup  B(W  n  Z  ■  z)  <  1. 

z*A  n 

It  follows  from  (2.13)  that  the  chain  is  positive  recurrent. 

For  the  final  example,  assume  that  the  transition  probabilities  for  the 
continuous  parameter  chain  (N(t):  t>0)  are  given  by: 


P(N(.t+h)  -  (x*l,y)  |N(t) 
P(N(t*h)  -  (x,y*l)  |N(t) 
P(N(t+h)  -  Cx-l.y)  |N(t) 
P{N(t*h)  ■  (x,y-l)  |N(t) 


(x,y)}  ■  ((l-a)x  ♦  By)h  ♦  o£h); 
(x,y)J  ■  (ax  ■»  (1-B)y)h  +  o(h); 
(x,y)J  ■  xh  +  o(h); 

(x,y)}  ■  yh  ♦  o(h). 
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This  example  was  proposed  by  Milch(1968).  Here,  the  numbers  a  and  6  satisfy 
0  <  a  <  1  and  0  <  3  <  1.  Note  that  (0,0)  is  an  absorbing  state.  At  each 
other  point  (x,y),  the  sum  of  the  birth  rates  is  equal  to  the  sum  of  the  death 
rates. 

Proposition  3,14.  The  embedded  Markov  chain  for  these  transition  rates  is  absorb¬ 
ed  at  (0,0)  with  probability  one. 

Proof.  Let  Zn  ■  ( Xn , Yn )  denote  the  state  after  the  n-th  Jump.  For  the 
function  4> C* J  ■  x  +  y,  observe  that  E($(Zn+j)  |Zn)  ■  $(Zn)  if  Zn  j*  (0,0). 

Let  Tq  denote  the  time  of  the  first  visit  to  (0,0)  of  paths  of  the  embedded 
Markov  chain  {Zn :  n>p).  It  follows  that  the  stopped  sequence  {0(2^^,) :  n>0} 
is  a  non-negative  martingale.  Thus,  it  converges  almost  everywhere.  Such  con¬ 
vergence  is  possible  only  if  the  stopping  time  Tq  is  finite  with  probability  one. 
This  completes  the  proof. 


mu  ini  whim 


Appendix  A.  Non-negative  Supermartlngales 

Let  (0,  F,  P)  denote  a  probability  apace  on  which  there  i»  defined  an  increas¬ 
ing  sequence  { Pn :  n>0)  of  sigma-fields  contained  In  P.  A  random  variable  is  an 
extended  roal -valued  function  defined  on  the  sample  space  fl  which  is  measurable 
with  respect  to  the  sigma-field  F.  Conditional  expectations  refer  to  the  prob¬ 
ability  measure  P, 

Definition  A.l.  A  sequence  {Wns  n>0}  of  non-negative  random  variables  is  a  non¬ 
negative  supermartingale  if,  for  n>0,  is  Pn-measurable  and  E(Wn+1 |F„)  <  Wn. 
Theorem  A, 2.  If  (Wn:  n>0)  is  a  non-negative  supermartingale,  then  the  sequence 
WR  converges  almost  everywhere.  The  limit  W  satisfies  the  inequality  B(W)  E(Wq). 

Note  that  the  inequality  B£W)  <  B(W0)  Implies  that  the  limit  is  finite  almost 
everywhere  when  the  expected  value  of  is  finite. 

Definition  A, 3.  A  random  variable  T,  taking  values  from  the  set  {ns  n>0)u(-*»}, 

is  a  stopping  time  if,  for  each  non-negative  integer  n,  the  set  {T>n}  belongs  to 

the  sigma-field  P  . 

n 

Definition  A, 4.  Let  T  be  a  stopping  time  and  let  f Wn :  n>0)  be  a  sequence  of  random 
variables  with  the  property  that  Wn  is  Fn-measurable.  The  random  variable  WT  is 
defined  on  the  set  {T<«»}  by  WT  ■  Wn  when  T  -  n. 

In  the  next  proposition,  the  notation  nAT  refers  to  the  stopping  time  ob¬ 
tained  by  truncating  T  at  the  integer  n. 

Proposition  A, 5.  Let  {WRs  n>0)  be  a  non-negative  supermartingale  and  let  T 
be  a  stopping  time.  Then  the  sequence  {W^s  n>0}  is  also  a  non-negative  super¬ 
martingale. 

Two  references  for  this  material  are  the  books  by  Doob  (1953)  and  Neveu  (.1975). 
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ABSTRACT.  We  have  programmed  a  "least  chi-square"  procedure  for  multiple 
independent  variables  to  analyze  time  series  -  especially  those  of  the  auto- 
correlated  moving  average  type.  We  compare  this  procedure  with  the  usual 
procedure  thsorltlcally  and  experimentally.  The  new  procedure  combines  in  one 
criterion  the  previous  method  which  usee  two  Independent  criteria  in  a  serial 
manner  -  first  one  and  than  the  other.  (These  two  criteria  axe  the  sum  of  squares 
of  the  residuals  and  the  Box-Pierce  test  for  their  randomness.)  Our  analysis 
of  three  well-known  series  illustrates  the  advantage  of  our  method.  The 
results  Indicate  that  it  obtains  a  more  probable  set  of  parameters  than  the 
older  method.  We  conclude  that  the  method  has  promise  in  simplifying  the' 
fitting  of  time  series.  By  having  a  single  measure  of  goodness  of  fit,  the 
latitude  for  variation  in  the  choice  of  fit  due  to  individual  judgment  appears 
to  be  reduced. 

I.  INTRODUCTION.  In  the  usual  methods  to  find  the  best-fitting  model  for 
a  time-series''”1"',  the  procedure  follows  the  schematic  shown  on  the  left  side 
of  Figure  1.  The  Investigator  chose  a  mathematical  model  as  an  initial  hypothesis. 
He  uses  a  computer  program  which  finds  the  unknown  coefficients  in  the  model 
using  the  principle  of  least  squares  to  estimate  the  beet  coefficients.  If 
the  least  squares  test  of  significance  indicates  a  reasonably  good  fit,  he  then 
computes  the  autocorrelation  coefficients,  forms  them  into  a  weighted  sum  of 
squares  of  autocorrelation  coefficients  (a  criterion  called  Box-Plercs  number) 
and  uses  the  resulting  figure  of  merit  to  test  whether  the  residuals  are 
consistent  with  sampling  from  a  series  each  of  whose  terms  is  selected  randomly 
(or  Independently)  from  an  error  population  which  has  a  normal  distribution. 

If  the  residuals  don't  pass  this  test  -  (or  any  of  the  other  tests  which  may  be 
used  for  the  same  purpose)  then  he  revises  the  mathematical  model  he  has  used. 

The  new  model  is  selected  to  reduce  those  values  of  the  autocorrelations  which 
are  largest.  In  this  way  he  uses  a  two  step  procedure t  First,  minimise  the 
squares  of  the  residuals  and  then  minimise  the  Box-Pierce  number.  (For  an  easy 
to  understand  review  of  this  process  see  Roberts (2)). 

On  the  other  hand,  one  of  us  (3, A)  has  shown  that  the  two  steps  can  be 
combined  by  adopting  a  combined  criteria »  The  sum  of  squares  of  the  residuals 
normalised  by  dividing  by  the  measured  or  estimated  variance  of  measurement, 

(this  has  a  Chi-Square  distribution.),  plus  the  Box-Pierce  criteria  (which  also 
has  a  Chi-Square  distribution,  if  the  residuals  are  independent).  In  this 
paper,  we  report  a  computer  program  developed  to  obtain  the  best  fitting 
coefficients  of  a  given  model  using  the  above  least  chi-square  criteria. 

Our  approach  to  fitting  time  series  is  to  use  this  program  to  help  determine, 
if  not  the  best  fitting  models,  at  least  a  good  fitting  model. 
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In  this  paper,  ws  will  provide  a  summary  of  tha  mathematics  usad  to  program 
our  cantral  CLC-6300  conputsr.  Wa  will  give  axamplas  of  data  fitting  on  thraa 
wall-known  sac  las  -  two  from  Box-Jar kins  (1)  and  ona  from  Hobart •  (2),  and 
compare  the  results  with  that  reached  by  tha  usual  approach. 

II .  MATHEMAT ICAL  FORKULAT f.ON .  Wa  lat  tha  mathematical  modal  be  such  that 
the  theoretical  values  are  raprasantad  by  tha  n  dimensional  vector  y*  and  tha 
independent  variables:  x,  u,  v,  w . 

y*  -  «0  +  «1*  +  *2U  +  *3V  +  ... 

Or  y*  -  $  x 


Wa  lat  y  be  tha  vector  of  observed  values  of  y*,  and  x*,  u4,  v*  be  values  of 
tha  appropriate  series  at  the  "1th"  observation.  We  will  refer  tha  reader  to 
previous  work  (4)  for  a  detailed  derivation.  In  that  we  differentiate  the 
criteria  CHI  SQ  TOT  (  Xy  )  with  respect  to  each  of  tha  unknown  coafficients 
and  set  them  to  tero  to  find  the  following  equation  in  matrix  form  for  the 
estimated  values  of  0,  denoted  as  0*. 

axT'/2 

— -  V  UP*  r  P0*)  -  p#  r  y>  ■  o 

where 

P'  •  1  1  1  1  .... 

*1  *2  *3  x4 

U1  u2  u3  u4 


... „ t. 27“j _ 

1  (d) '(«/•»  -21,  (<•,)»/», 

a  [m|  i  I 


r  ■  T  +  z  o  [  V  “l  +  V  "M 
1  |«|  L  Vi  -I  J 

I  is  the  n  dimensional  unit  matrix.  V*7^  is  tha  matrix  resulting 

when  wa  shift  tha  columns  of  1  by  "i"  columns  to  the  right  and  insert  seros  in 
the  columns  which  remain,  is  tha  "1”  th  autocorrelation.  9  t  is  the 

variance  of  the  measurement  error,  d  is  tha  vector  of  the  residuals?  is 

the  variance  of  (r  .  On  multiplying  out  ?*  r  and  [P‘  T  P]  wa  find  the  usual 
least  squares  expression  as  Rivpa  in  Figure  2A  and  2B,  but  modified  by  tha 
addition  of  the  terms  which  contain  <j  in  their  expression. 

All  terms  except  the  value  of  the  q  s  can  be  evaluated  at  once  from  the 
data  input.  We  assume  initially  each  is  zero;  calculate  them  irom  Che 


LEFT  SIDE  OF  REGRESSION  EQUATION 


EIGHT  SIDE  OF  REGRESSION  EQUATION 
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autocorrelations  of  tha  residuals,  and  Iterate  to  find  the  new  values  of  the 
vector  0*.  We  usually  chose  an  incremental  decrease  of  11  or  less  in  the 
value  of  CHI-SQ  TOT  to  stop  iteration;  however  this  criteria  is  an  input 
parameter  on  each  run.  The  significance  of  the  value  of  CHI  SQ  TOT  has  been 
calculated  from  the  formula 


Significance  ■  V2x^T 


r2(n  +  s  -  p)  -  1 


where  p  is  the  number  of  parameters,  and  s  is  the  number  of  autocorrelations 
used.  This  is  well-known  to  have  a  normal  distribution  with  a  mean  of  zero  and 
a  standard  deviation  of  1.  It  will  be  used  to  compare  the  results  of  the 
computations  using  the  same  initial  data  but  possibly  different  numbers  of 
data  points,  autocorrelations  and  parameters.  If  the  "significance"  of  a 
given  model  has  a  larger  positive  value  then  another,  it  is  less  significant. 

III.  EXAMPLES  OF  TIME  SERIES.  In  order  to  provide  the  reader  with  some 
readily  available  data  to  which  we  have  applied  our  technique,  we  used  two 
series  supplied  by  Bom  and  Jenkins  (1).  Figure  3  shows  a  plot  taken  from 
Figure  4.1  of  reference  (1).  We  have  analyzed  both  Series  C  and  Series  D. 

In  addition  ve  have  analysed  the  GNP  data  of  Roberts.  We  will  begin  with  the 
Series  C. 

III-l.  TIME  SERIES  C.  In  Figure  3  this  series  appears  quite  smooth  -  from 
this  ve  might  expect  that  a  rather  long  term  lag  might  be  present  in  addition 
to  short  term  fluctuals  (and  indeed  we  found  such  a  long  term  lag).  We  had 
no  prior  knowledge  of  the  error  of  measurement  for  this  series.  We  can, 
nevertheless,  estimate  the  lowest  possible  value  of  this  error.  We  do  this 
from  the  observation  that  only  a  simple  decimal  point  accuracy  is  carried  in 
the  data  analysis.  The  digitising  error  -  assuming  a  uniformly  distributed 
population  -  is  estimated  as  .029.  Later  in  the  process  we  examined  the 
sensitivity  of  the  results  to  this  assumption,  by  multiplying  the  error  by 
five  thus  increasing  it  to  .145. 

Turning  now  to  our  analysis,  the  autocorrelations  in  the  four  cases  selected 
to  illustrate  our  results  are  listed  in  Table  1.  The  first  step  in  fitting 
this  series  was  to  analyse  the  data  in  terms  of  the  first  and  second  lagged 
series.  The  results  as  Indicated  for  Case  10  in  Table  2  (Series  C  Results) 
indicate  a  rather  low  level  of  significance  (73.8)  (see  section  II  for  the 
definition  of  "significance")  with  the  largelt  part  of  CHI  SQ  total  coming 
from  the  sume  of  squares  of  the  residuals.  The  largest  terms  in  the  auto¬ 
correlations  are  of  rank  9,  10,  11,  and  12.  (See  Table  1.)  The  next  step, 
we  computed  case  14,  with  the  estioutad  variance  increased  by  a  factor  of  5. 

This  reduced  the  value  of  CHI  SQ  1  to  180.1;  the  value  of  CHI  SQ  2  Increased 
basically  due  to  the  Increase  in  the  number  of  autocorrelations  from  12  to 
20.  The  autocorrelations  of  rank  9,  10,  11  and  12  remained  the  largest. 

Case  1  in  Table  2  shows  the  effect  of  changing  the  model  to  using  a 
differenced  Independent  variable  and  the  first,  eighth,  and  ninth  lagged 
series  as  predictors  (using  again  the  small  value  (.029)  of  the  "error  of 
measurement").  The  value  of  CHI  SQ  2  is  substantially  reduced;  and  the  sum 
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FIGURE  3 

SOKE  TYPICAL  TIKE  SERIES  ARISING  IN  FORECASTING  AND  CONTROL  PROBLEMS 
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of  equate*  it  ellghtly  reduced.  The  9th,  10th,  tad  12th  autocorrelation* 
(Table  1)  are  alightly  raducad,  but  tho  11th  autocorralatloa  1*  increased. 

The  "significance"  la  improved}  alaea  tha  value  daeraaaad  from  73.1  to  71.9. 

If  thla  ease  caaa  la  run  without  tha  8th  and  9th  lag*  the  valuaa  for  case  4, 
Indicate  that  tha  CHI  SQ  TOT  increaaea  a uba cant tally,'  even  above  tha  Caaa 
10.  Thla  Indicates  the  elgniflcaace  of  tha  coefficients  3  aad  4  for  Caaa  1, 
since  chat  la  tha  only  dlffarasea  between  tha  two  caaaa.  In  Caaa  4,  the  auto¬ 
correlations  of  rack  9,  10,  11,  and  12  are  all  comparable  with  Caaa  10. 


Comparison  with  Box-Jeoklna.  Caaa  4  ecn  ba  coopered  with  one  of  the 
reaulca  of  Bom- Jenkins.  In  thalr  terminology  thla  series  waa  given  aa  a 
alagla  difference  la  th»i  tine  aarlaa  «t,  (vat).  **  *  function  of 

tha  once-lagged  difference  f,  .  ,  and  ena  reaiduala  a.  ♦  They 

found  e"1  * 


**t  ’  *t+  •*2f«t.l 

In  our  laaat  aquarea  aaalyala  ve  found  that  aa  additional  conataat  value  of  ,00769 
eould  be  added  to  tha  right  of  thla  taro,  aad  chat  tha  eoafficlant  of  7lt.i  vaa 
,8133.  Using  LCS  theae  valuaa  vara  changed  only  ellghely.  The  raault  which 
ve  have  obtained  pravloualy  that  Caaa  1  la  aore  algnlf leant  than  Caaa  4  applies 
equally  well  to  the  conolualon  that  Caaa  1  la  oora  aigniflcant  than  the  »ojc- 
Jenlclna  raault.  In  Box- Jenkins  teroa  our  raault  let 
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We  atopped  the  analyela  of  thla  aerlea  at  thla  point,  although  ve  night 
have  well  conaldered  the  addition  of  a  lag  of  10,  and  poealbly  11  to  the 
lagged  varlablea  uaed  In  Caaa  L,  becauae  of  the  rather  large  valuaa  of  the 
auto correlation*  of  rank  10  aad  U  la  Table  1. 

III-2.  31*113  D.  Aa  la  the  caae  of  Serlea  C,  ve  Initially  uaed  two 
lagged  aerlea  to  determine  the  regreealon  of  the  aerlea.  We  eee  from  Table  3 
that  all  the  autocorrelatloae  fron  1  to  20  ere  leaa  than  .1,  however  ve  find 
that  If  ve  uee  only  one  lagged  parameter  our  fit  la  beteer  elnce  tha  elgniflcaace 
la  better.  (In  caae  27  ve  uaed  fever  teroa  than  la  Caae  1,  eo  that  a  direct 
coaparlaon  of  the  value*  of  CHI  SQ  TOT  would  be  ala lea ding. )  When  ve  added  two 
differenced  parameter*  to  Che  calculation,  aa  la  Caaa  5,  ve  found  aa  lacreaae 
la  the  value  of  CHI  SQ  2,  aa  Indicated  by  tha  Increeaed  value  of  tha  2nd  to  4th 
autocorrelation  eoafflclanta.  Other  chaagee  have  been  aada  hare,  the  auebar 
of  autocorrelation*  ha*  been  reduced  eo  ala  from  20,  end  the  aaaunad  aaaaurement 
error  haa  been  reduced  to  .029  fron  .030.  It  la  surprising  that  the  use  of 


lagged  difference  independent  variables  does  not  Improve  tha  fit.  On 
Increasing  tha  expected  measurement  error  by  a  factor  of  tan  whan  using 
one  lagged  and  one  differenced  variable,  as  In  Cases  6  and  8,  we  find  a 
slight  reduction  In  CHI  SQ  2  since  the  program  puts  more  emphasis  on  It  In  such 
a  case.  On  setting  tha  mean  arbitrarily  to  zero  (l.e.  coefficient  1  Is  zero, 
as  In  Casa  8)  wa  find  we  can  make  tha  usual  argument  that  the  number  of 
adjustable  parameters  Is  leas  than  before.  We  Interpret  the  reduction  ol' 
significance  In  Casas  1,  5,  and  6  (as  compared  to  2,  and  8)  as  due  to  the  fact 
tha  additional  terms  add  "noise"  to  tha  theoretical  value.  Similarly,  we 
attribute  the  relative  increase  of  CHI  SQ  2  from  Case  2  to  Case  8,  to  the 
Increased  noise  introduced  by  using  the  first  difference.  Case  2  la  an 
approximation  to  tha  result  of  Box- Jenkins  who  found: 


z„  ■  .87z„  .  +  1.17  +  a„ 

t  t-1  t 

A  subaet  of  the  basic  data  was  used  so  that  the  difference  between  the 
coefficients:  .87  vs  .85  and  1.17  vs  1.35;  is  not  surprising. 

III-3.  08088  NATIONAL  PRODUCT.  The  data  we  used  in  our  studies  were 
given  by  Roberts  In  Column  1  of  his  Table  11-1.  We  chose  this  sat  to  analyze 
because  Roberts  has  given  a  detailed  analysis  of  these  data  which  Is  easy  for 
anyone  to  follow  and  with  which  wa  can  make  a  simple  but  meaningful  comparison. 
Our  first  step  in  the  discussion  is  to  present  and  compare  least-square  and  LCS 
computations.  We  have  repeated  Roberts  LS  analysis  and  used  the  results  both 
as  validation  of  our  computer  program  and  as  a  basis  of  comparison  between 
Roberts'  results  and  ours. 

To  show  the  difference  between  the  least  square,  and  tha  least  chi-aquara 
results,  we  have  prepared  Tables  5  and  6,  Consider  the  residuals  In  Table  5 
for  four  cases  of  interest:  1  through  4.  In  Case  1,  the  autocorrelations 
of  orders  of  2,  4,  6,  and  11  are  large  (greater  than  .1).  In  Case  2,  rank 
6  is  reduced  but  5  and  7  are  increased.  On  going  to  Case  3,  ranks  2,  4,  5,  7 
and  11  are  still  large  and  the  Box-Fierce  coefficient  is  larger  than  Case  2. 

For  Case  4,  ranks  2,  3,  and  8  are  large,  but  the  Box-Fierce  coefficient, 

(CHI  SQ  2)  has  become  less  than  half  of  Case  1. 

In  Table  6,  Cases  1-4  are  given  to  compare  least  squares  (LS)  and  least 
chi  square  (LCS).  In  all  cases  of  LCS  tha  value  of  the  sum.  of  squares  of  the 
residuals  is  slightly  increased  thus  increasing  CHI  SQ  1.  On  tha  other  hand, 
in  every  case  CHI  SQ  2  is  substantially  dscreaaad,  and  results  in  a  decreased 
value  of  CHI  SQ  TOT.  The  regression  coefficients  for  the  lagged  variables 
change  in  all  cases  and  occasionally  change  by  a  substantial  amount,  ee 
would  expect  on  going  from  the  LS  to  the  LCS  analysis.  The  column  labelled 
"L"  in  Table  6  indicates  the  amount  of  lag  which  has  ganarated  tha  independent 
series  which  corresponds  to  the  coefficient  given  in  the  column  next  to  it. 

Baaed  on  results  of  the  first  case  the  variable  lagged  by  7  (corresponding 
to  coefficient  4)  was  replaced  by  tha  aeries  which  lagged  by  8.  The  difference 
between  Case  2  and  Case  3  is  that  in  the  former,  the  error  was  assumed  as  .06 
while  the  latter  It  was  assumed  as  .04.  Thus  Case  2  puts  more  weight  on  the 
reduction  of  CHI  SQ  2  than  does  Case  3,  and  the  results  indicate  the  same. 

The  regression  coefficients  are  little  different  in  Case  3  from  those  in  Case  2. 
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TABU  5  AUTOCORRELATIONS  FOR  VARIOUS 


Xn  Cm*  4,  the  only  change  from  Cast  2  was  eha  uaa  of  tha  9th  laggad 
•arias  la  placa  of  tha  saeond  laggad  sarias.  This  changa  was  mada  bacauaa 
tha  coefficient  of  tha  tvlea  laggad  sarias  was  small;  wa  raplacad  tha  twice 
laggad  sarias  with  tha  9  times  laggad  sarias  to  attampt  to  raduca  tha  large 
autocorrelation  at  ranks  4  and  larger,  which  none  of  the  previous  models  had 
bean  able  to  do.  This  changa  made  a  big  improvement  (from  Case  2  to  Case  4). 

Since  tha  coefficient  of  tha  sixth  term  is  now  small,  whan  wa  continue, 
wa  will  eliminate  it  and  test  to  sea  if  uaing  tha  twice-lagged  sarias  might 
reduce  tha  second,  third  and  ninth  autocorrelations. 

Xn  order  to  compare  with  Roberta*  analysis  of  GNP,  wa  took  eha  4th 
difference  of  tha  first  difference  and  obtained  a  dependant  variable  which 
is  "twice"  differentiated".  (Tha  autocorrelations  for  Case  13  are  not  shown 
in  Table  3  since  they  were  similar  to  those  of  Casa  14.)  >  Of  tha  four  cases 
analysed  (refer  to  Tables  5  and  7)  tha  autocorrelations  and  tha  Box-Pierce 
number  (CBX  SQ  2)  of  Casa  11,  are  smaller  than  those  of  tha  others.  Xn  12, 
the  second  laggad  variable  was  omitted,  and  the  significance  test  applied. 

The  results  indicate  that  Case  11  is  slightly  more  significant  than  Case  13, 
and  more  significant  than  Case  12. 

Case  13  was  used  as  a  base  case  to  find  the  effect  of  varying  the  "erf or" 
or  estimated  precision  of  measurement  of  the  transformed  GNP.  The  number  .06 
corresponds  to  an  estimate  of  a  12  error  in  the  measurement  of  the  GNP. 
Correspondingly  Case  14  corresponds  to  an  approximate  error  of  I*j2  in  the 
GNP.  The  estimated  measurement  error  was  introduced  in  a  progression  of 
computations  going  from  .03,  .06,  .12,  .24,  to  .48.  The  "deviation"  from 
the  expected  value  was  plotted  on  Figure  4,  as  a  function  of  the  asaumed 
error.  An  Iterative  procedure  was  used  to  interpolate  on  the  curve  around 
the  "0"  deviation  value;  Case  14  was  the  result.  We  infer  from 
this  curve  chat  approximately  .093  is  the  "best"  estimate  of  the  standard 
deviation  of  the  measurement  of  the  GNP. 

When  we  compare  the  results  of  the  analysis  of  the  twice  differentlaled 
series  of  Case  11  with  that  of  the  single  differentiated  series  of  Case  4  we 
observe  that  Case  4  has  a  smaller  value  of  the  significance  parameter  6.42, 
as  compared  with  6.89.  Thus  if  the  assumptions  behind  these  two  cases  are 
valid,  we  would  conclude  that  the  use  of  the  single  differenced  procedure  is 
better  than  the  twice  differenced  procedure.  However,  we  prefer  to  conclude 
that  there  is  no  real  difference  between  the  two.  This  conclusion  is  subject 
to  a  further  caveat  that  the  comparison  depends  crucially  on  the  value  of  the 
measurement  error  assumed;  as  we  have  seen  from  our  previous  example,  its 
expected  value  is  about  .093.  Further  study  of  this  matter  is  clearly 
indicated.  The  least  squares  analysis  (Case  13)  may  be  used  to  compare  the 
Roberts'  analysis  of  GN?  with  ours.  Roberts  found  coefficients  of  (0,  .1261, 
.2393,  -.38);  in  our  first  iteration  we  found  (.0033,  .1242,  .2362  and 
-.3712).  The  CHX  SQ  TOT  was  13.68,  and  the  significance  is  7.02.  Comparing 
this  result  with  our  final  iteration  of  Case  13,  we  soe  that  the  significance 
is  decreased  by  .10.  Comparing  with  our  best  fit  "Case  4",  we  found  an 
increase  of  "significance"  from  6.42  to  7.02.  Thus  we  see  our  procedure,  using 
the  six  lagged  series  gives  a  better  fit  since  its  "significance"  is  smaller. 
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IV.  C0HCIP8I0M.  Cotaparlaon  of  tha  loose  chl-aquara  (LC8)  analyst*  of 
thraa  vall-knovn  tlaa  aarlas  with  tha  usual  laaae  aquaraa  analysis  lndlcataa 
that  la  ovary  eaaa  a  more  significant  aat  of  paranetera  la  obtalaad  by  tha 
LCS  aaalyala.  Wa  think  that  thia  procadura  should  ba  atudlad  la  aora  datall 
to  further  valldata  this  conclusion. 
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AN  APPLICATION  OP  RENEWAL  THEORY  TO 
SOFTWARE  RELIABILITY 


Leonard  A.  Stefanski 
U.S.  Army  Reeaarch  Off lot 


ABSTRACT 

In  this  paper  we  introduce  n  probabilistic.  model  for  describing  software 
failures  which  generalizes  several  models  appearing  in  the  literature.  Associ¬ 
ated  with  this  model  is  a  Superimposed,  Delayed  Transient  Runeual  Prooese, 
(SDTRP) .  The  structure  of  such  processes  is  oxplolted  to  obtain  several  quan¬ 
titative  measures  of  software  performance.  In  addition  wo  aro  able  to  consoli¬ 
date  the  literature  on  software  reliability  by  pointing  out  that  several  models 
are  special  cases  of  SDTRP' s.  Finally,  some  results  relevant  to  inference  and 
goodness-of-fit  tests  for  a  restricted  class  of  models  ore  presented. 
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0.  INTRODUCTION 


In  tho  past  decade  novornl  studios  of  tho  siochaatic  bohnvior  of  computer 
software  failures  Imvo  been  undortnkon  with  tho  objoctive  of  developing  analyti¬ 
cal  models  to  be  used  to  obtain  quantitativo  measures  of  software  performance. 

H or  the  most  part  the  dovolopiuont  and  analysis  of  these  iitodols  has  procoodod  along 
highly  individualistic  linos,  This  lias  Jod  to  somo  duplication  of  offort  and,  more 
importantly,  has  precluded  a  cohoront  study  of  softwuro  reliability  modeling.  This 
pnpor  at  tempts  to  remedy  this  situation  by  codifying  a  snirill  hut  important  sogmont 
of  the  literature. 

In  Section  i  we  describe  a  scenario  for  tho  generation  of  software  failures 
which  leads  us  to  consider  suporimposod  delayed  transient  renewal  processes  as 
models  of  software  reliability.  Several  measures  of  suftworo  porformanco  are  thon 
obtained. 

In  Section  2  wo  show  how  this  model  goneralisos  the  works  of  Jolinski  and 
Morundn  (11)72),  L.lttiowood  (11)81),  and  Duel  and  Okumoto  (1D78).  Wo  also  point  out 
an  lntorosting  relationship  between  the  above  models  and  that  of  Gool  and  Okumoto 
(1D79). 

Pinal ly,  Section  5  contains  results  relevant  to  Inforonco  and  goodness-of-fit 
tosts  for  the  class  of  ordor  statistic  models.  Those  arc  statistical  questions 
which  hnvo  not  boon  adequately  addressed  in  tho  past,  for  Jolinski  and  Morandn's 
mod'.*  1  tho  uso  of  maximum  likelihood  to  ostium  to  parameters  oft  on  yields  usoIokh 
results,  Tho  prohlom  is  shown  to  ho  equivalent  to  estimating  population  si, so  when 
observations  aro  obtained  by  truneatod  sampling,  thus  the  work  of  lUumonthul  and 
Mumis  (1975)  can  ho  brought  to  hear  on  t.hic  subject.  In  addition  a  procedure  for 
comparing  tho  fit  of  those  modoJs  to  existing  data  sots  is  outlined,  This  is  illus¬ 
trated  with  dutu  from  n  software  dovolopmont  project. 
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1.  MODEL  DUVBLOFMUNT 


A  software  program  may  contain  errors  yet  still  be  capable  of  successful 
execution  for  certain  types  of  inputs.  A  failure  occurs  when  an  input  is  processed 
which  causes  one  of  these  errors  to  manifest  itself.  Since  debugging  takes  place 
we  expect  the  number  of  errors  to  docrcasc  and  program  reliability  to  increase. 

With  this  in  mind  we  offer  the  following  interpretation  of  the  software  failure 
process. 

Let  N  represent  the  initial  number  of  errors  in  the  software  and  for  ease  of 
reference  call  thesa  Ej,  B2,...,EN>  Furthermore,  assume  that  the  detection  of 
is  independent  of  that  of  Ej  for  i  i  j.  Let  us  focus  attention  on  just  one  of 
the  Bj's,  say  E  and  let  M(t)  count  the  number  of  software  failures  experienced  in 
[0,t]  due  to  E.  is  the  time  until  the  first  of  those  faiiuros  and  has  distribu¬ 
tion  P(x)  ■  PrCXj  <  x).  If  errors,  when  detected,  are  corrected  with  probability 
one,  then  M(t)  is  a  very  simple  point  process,  assuming  only  the  values  0  and  1, 


0  s  t  <  X, 

i 

t  a  X, 


M(t)  is  a  Bernoulli  random  variable  with  Pr[M(t)  ■  1]  ■  P(t).  However,  if  debugging 
is  not  successful  then  we  will  experience  a  second  failure  after  some  timo  X2.  A 
reasonable  assumption  is  that  Xj  and  X^  are  independent  and  identically  distributed. 
Similarly,  if  the  first  k  attempts  at  correcting  E  are  unsuccessful,  we  would  reason 
that  the  inter-fniluro  times  X^.Xj,. . . ,X^+ ^  are  i.i.d.  with  common  distribution  P. 
If,  however,  errors  are  corrected  with  probability  p,  0  <  p  <  1,  there  is  positive 
probability  that  the  number  of  software  failures  caused  by  ti  terminates  at  some 

finite  value  n  *  1,2 .  In  particular,  if  Yj.Yj,...  are  the  inter-fniluro  times 

of  the  M(t)  process  under*  impevfaat  debugging  thou 


- . . . . . 


X£  with  probability  q 

+»  with  probability  p 

^  Xj  with  probability  q 

■*  '  w  with  probability  p, 

where  q  »  1-p.  Thus  Yj  is  a  proper  random  variable  having  distribution  F  and  for 
j  i  2  Yj  is  a  defeotive  random  variable  with 

Pr[Yj  st]  ■  qF(t)  0  s  t  <  » 

Pr[Yj  ■  +»]  ■  P  . 

The  M(t)  process  is  a  delayed  transient  renewal  process. 

Let  Sj  ■  Yj+.o+Yj  be  the  time  of  the  j-th  failure  due  to  E.  Then  for  any 
finite  tiOwe  have  Pr(Sj  st)  «  PrCY^,  ,,+Y.j  s  t) 

■  Pr(Yj+, . , +Yj  s  t,  and  Yj  ■  i-1,... J) 

•  Pr(X1*...+Xj  s  t)  n  P(Xi  ■  Yt) 

“qJ"lp(j)(t)  a-1) 

where  is  the  j-fold  convolution  of  F  with  itself.  In  deriving  (1.1)  it  has 

been  tacitly  assumed  that  the  debugging  attempts  are  independent  of  one  another 
and  of  the  occurrence  of  failures.  Since  [M(t)  a  k] =  [Sk  s  t]  we  find  that 

Pr(M(t)  *  k)  -  qk"1F(.k)  (t> 

and  therefore  the  renewal  function  H  (t)  is  given  by 

OO  00 

H  (t)  -  ElM(t)J-  l  Pr[M(t)*k]  -  [  qk'1Frl|,(t).  (1.2) 

M  k-1  k»i  '■*' 
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One  thou  obtains  the  following  relationship 


HqCt)  »  F (t)  ♦  qCFniq)(t) 


(1.3) 


where  *  denotes  Lebesgue-Stieltjes  convolution.  Letting 


end 


h  (s)  -  /  e~stdH  (t) 
M  0  M 


f°(s)  ■  /  e‘8tdP(t) 
0 


the  relationship 


h>  -  -jy* 

’  I-qf  (s) 


we  have 


(1.4) 


which  can  be  used  to  determine  1^  (t)  given  any  distribution  F,  The  probability 
generating  function  of  M(t)  is  defined  as  4>M^(s)  ■  B(oxptM(t)log  s]}  and  it  can 
be  established  that 


'  ♦M(t)00  "  1  +  (S_1)  “c|SCt)' 


0  <  s  <  1. 


Now  let  J(y)  *  Hy(t)  for  0  <  y  <  1  then  we  have 


and 


^M(t)(s)  "  1  +  (8‘^  JC‘(s) 

■l(k)(M»)^qk“l  (q»). 


T*(t) 

jk 

d#,  tvM(t) 

rrhus  the  k-th  factorial  moment  of  M(t)  is  given  by 

,k 

G{M(t)  (M(t)  - 1  ]•...♦  |M(t)-k+  lj )  =  d\<f>M(tJ(s) 


-  fcqk-1  .|Ck-0(,p, 


3-1 


and  the  variance  of  M(t)  is 


Var[M(t)]  ■  2q  J  (q)  +  J (<|)  -  [J(q)l2 


Finally  note  that  M(“)  is  a  negative  binomial  random  variable  with 
Pr{M(»)  -  k}  -  pqk_1  k-1,2 . 
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A  random  variable  of  considerable  interest  is  W,  the  time  until  error  E  is 

k- 1 

corrected.  Clearly,  W  =  Xj+.-.+X^  with  probability  pq  and  hence 

CU 

P[wst)  -  l  pqn-1Pfnat)  -  pH  (t). 
n-l  inj  q 

This  is  a  proper  distribution  function.  Wo  can  find  the  mean  and  variance  of  W  by 
noting  that 

03  00  00  .  H 

N(W)  -  l  pq°" 1  /  tdP.  »(t)  ■  £  pq  IIR-  — 
n-l  0  '•nj  n-l  1  p 

and 

,J1  2 

Vnr(W)  «  12 (IV  -  -—) 

“n ft‘rj2  u,,(n)(t) 

•  ?Pqn_1[n02+M2(n-i)2] 
n-l  p 

2 

whero  and  a  nro  the  moan  and  vnrinnce  of  Xj. 

A  natural  assumption  is  that  each  of  the  N  errors  gives  rise  to  a  SDTRP,  not 
necessarily  stochastically  identical.  In  particular  we  assume  that  error  E^, 
i»l,...,N  has  associated  with  it  the  following  quantities: 


a)  F  ■  Distribution  of  t.he  time  until  first  failure  due  to  E^. 


by  »  Probability  of  correcting  12^  on  any  trial,  ■  1  -P^ . 


cj  M^(t)  ■  Number  of  failures  in  [0,t]  due  to  .  Pr{M^ (t)*k}«  (q^^pi  (t) 


00 


00 

d)  iij  (t)  -  BiMjit)]- iIi(qi3k'1n[k)(t) 


k-l 


00 


e)  J^y)  -  Hj(t). 

f)  ^(s)  -  <J>M  ^(s)  *  Probability  generating  function  of  M^(t). 

g)  -  Time  until  12^  is  corrected.  PrlW^t)  *  (t). 
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tewuumu . «nwnf  i,Wi»i!l<..«»... 


uan;  -"Tirmii^Ti 


Now  let  M*(t)  count  the  number  of  failures  oj'  all  types  experienced  in  [0 , t J . 

Then  M*(t)  =  M,  (t)  +  ...+MM(t)  and  hence  ll(t)  =  l:lM*(t)]  =  II*  (t)  + . . .  (t) . 

1  N  ql  qN 

Also,  sinco  we  have  assumed  the  detection  of  li.  to  be  independent  of  that  of 

N  N  ,  2 

E,,  ij*j ,  we  also  have  Var[M*(t)]  ■  J  Var[M,(t)]  ■  ^{2q  Jj(q,)  +  JjCqJ  - 
3  i-1  N  i=l  1 

and  E{exp(M*(t)log  s)}“Vt)(s)  ‘  ^  V8)l 

1.1  Measures  of  Software  Performance 


We  are  now  in  a  position  to  derive  the  distributions  of  1)  T,  tho  time  to  a 

completely  debugged  system,  2)  T  ,  tho  time  to  a  specified  number  (n)  of  remaining 

errors,  and  3)  X(t)  the  number  of  errors  in  the  program  at  time  t.  We  will  make 

use  of  the  ordered  random  variables  W/n<  W, .<W,.M  whore  W,,,  is  the  i-th 

(0  (2)  CN)  (i) 

order  statistic  from  (W^Wji  >  »WN) . 

The  software  is  completely  dobugged  if  and  only  if  each  of  the  N  errors  has 
been  corrected.  Thus  {T  s  t}  iff  {W^s  t}  i«l,.,.N.  And  we  find 

Pr{T  s  t)  •  Pr{max{W^ , . . . ,W^}  s  t} 

"  |,r{W(N)5 


N  l 

■  n  (p,  Hl  co),  t  >  o. 

J  _  1  M  l 


i-1  ^  ql 

Economic  and/or  time  considorations  may  make  us  willing  to  tolerote  an  upper 
bound  n  on  the  number  of  errors  remaining  in  the  program.  Thus  the  distribution  of 
T  is  of  interest.  The  event  {T  s  tl  occurs  Iff  at  least  N-n  errors  have  been  cor- 
tected  by  time  t.  Thus  we  have 


Pr{T  .<  t)  -  PrfW  ..  .st), 

n  (.N-n) 


Proceeding  similarly  one  finds  that  {X(t)  a  n}  if  and  only  if  {W^  n+1^>t}. 
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Titus 


l‘r{X(t)  *  n)  -  Pr{WfN_0+1)>  t) 
and 

MX(t)  »  n)  •  HrtWCN-a)si  t)  -  l’’r<W(N>.ll4>i)s  t3, 

also 

iHX<t)]  t)  . 

Homin' ka:  lui  lu'bUl'tir/  cliuiguu  gl'  I* . I'N  unii  tlioag  formuluu  ftro  qutto 

12  N 

cumbersome.  However,  It  is  often  assumed  that;  I’j  •  Dj*..."  1^  unci  F  ■  I'  F 

In  which  ease  tiro  Identically  iih  well  u:i  i ndoporidontly  distrihutod.  re¬ 

pressions  Cor  the  joint  distribution  of  order  stntint  Ics  lire  well  known  for  thu  l.l.tl. 
ease.  In  fact  if  the  common  distribution  function  is  P(x)  ■  1  -  exp(-Xx)  then  tho 
Wj 1  s  have  distribution  l’r{W.  s  t)  •  1  •  oxp(-Xpt)  and  tho  ubovo  expressions  sim¬ 
plify  consldorubly. 

2.  MO DHL  CONSOLIDATION 

■  ■■■■■— 1«—  ..l,  »i  I  HIM  ■■  •  I. 

Consider  tho  simpl  i  fienvion  obtained  by  taking  ■  1*^  i*^“  P  and 

l1^  ■  F2  I,N«  I',  If  wo  further  restrict  P(x)  ■  1  -  o’*x  we  imvo  the  modol  an¬ 

alysed  by  (Joel  and  Okumoto  (11173)  .•  Their  model  in  turn  is  a  generalization  of  that 
of  Jollnskl  and  Mornnda  (11172)  to  Include  the  possibility  of  Jmporfoct  debugging. 

Wo  will  show  that  tiio  model  pro|>osud  hy  l.ittlowood  (1081)  U  of  tho  saaio  "typo"  as 
tho  J-M  model.  To  allow  for  Imporfoct  debugging  under  Llttlewaod'a  framework  wo 
noud  only  take  Pj  ■  V  and  Pj  ■  «...■  IVN  ■  P  whoro  F(x)  •  1  - 

In  addition  to  simplifying  the  analysis  in  studying  tlieso  models  (the  primary  obstacle 
is  finding  ll^(t))  the  duvolupiuent  in  terms  of  Sl/ruP • »  provides  an  alternative  inter¬ 
pretation  of  tho  failuro  process. 

Uuo  of  the  curl  lust  and  certainly  must  referenced  modois  of  software  failures 
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ii  that  of  Jolinski  and  Moranda  (1D72).  Our  fromowork  includes  this  under  the  re¬ 
strictions  I’j  ■  I'j  1'^  *  I  mid  lfl  *  P2  ,=  PN  a  |J  =  1  -  c"Xx.  Recall  that 

for  the  coso  of  porfoct  debugg l ng ,  (P  ■  11  each  orror  gives  rise  to  a  Bernoulli 
point  process,  M^(t)  ■  0  if  hss  not  boon  corrected  by  time  t,  Mj(t)  ■  1  otherwise. 
Thus  if  the  M^s(t)  aro  l.i.d.  thon  M*(t)  «  M1(t)  +  ...+MN(t)  has  a  binomial  distri¬ 
bution; 

Pr{M* (t)  -  k}  -  (J)[P(t)]k[l  -  P(t)]N‘k,  k«l , . . . ,N. 

M* (tj  is  colled  nn  Order  Statistic  process  since  the  time  of  occurrence  of  the  j-th 
event  is  distributed  as  tho  j-th  order  statistic  from  a  population  of  site  N  with 
common  distribution  function  P.  .lelinski  and  Morandn  orininnlly  specified  tho  joint 
distribution  of  n  i  N  whore  Y^  is  the  time  between  discovery  of  the 

(i-l)-st  and  1-th  orror; 

n  -X(N-i+l)Y. 

f(y,...yj  ■  n  (N-iel)Xo  \  (2.1) 

1  n  1-1 

An  easy  computation  establishes  that  tho  random  variables  ■y1+...+Y. 

i«l,...,n  are  indeed  the  ordor  statistics  from  an  exponential  population.  This 

observation,  until  now  overlooked,  has  important  implications  when  it  is  desired 

to  estimate  the  unknown  parameters  N  and  X.  This  problem  is  addressed  in  Section  5. 

He  stated  oarlier  that  the  model  of  Littlcwood  (IH81)  is  the  same  "type"  as 

that  of  Jolinski  and  Morandn.  In  fact,  Littlowood  has  characterised  an  order  statistic 

process  generated  by  n  Pareto  population.  ‘Hint  Is,  X^,  tho  timo  of  occurrence 

of  the  j-th  failure.  Is  distributed  ns  tho  j-th  ordor  statistic  from  a  population 

S  ct 

of  size  N  with  common  distribution  P(x)  *  l  -  (p-^— ) 

Order  statistic  models  can  bo  motivated  via  tho  following  argument.  Assume 
there  are  N  errors,  11^,..., li^.  Let  bo  tho  debugging  time  needed  to  reveal 
Then  the  time  of  discovery  of  tho  first  error  is  just  the  first  ordor  statistic 

from  Xj,. . .  ,XN>  Similarly  for  j»l,...,N. 
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In  a  Inter  paper  (Joel  and  Okumoto  (11)79)  surest:  model  I  ng  software  error  de¬ 
tections  via  a  non-homogoneous  Poisson  process.  They  choose  a  mean  function  of  the 
form  (J (t)  a  a(l-e  *’*).  This  model  i;.  closely  related  to  the  order  statistic,  pro¬ 
cesses  as  the  following  theorem  demonstrates. 

Theorem  1  i  If  for  a  Poisson  process  it  Is  given  that  exactly  N  events  have  occurred 
in  [0, tg],  then  the  event  times  are  distributed  ns  order  statistics  from  the  distri¬ 
bution  F(x)  »  ,  0  s  x  s  tQ. 

Proof ;  See  Thompson  (1981)  for  a  proof  of  this  theorem  and  related  results. 

To  apply  this  result  in  tho  prosont  context  take  tQ  -  »,  (l.e.,  t^  «) .  Then 
conditioned  on  the  event  that  exhaustive  debugging  uncovers  a  total  of  N  errors, 
the  times  of  error  discoveries,  X^  <  Xf2^  <.,.<  X(N),  are  order  statistics  from 
the  distribution  F(x)  ■  ■  1  -  e"bx.  Tho  resulting  failure  time  structure  is 

identical  to  that  of  the  Jel inski -Moranda  model.  It  is  also  pdssible  to  assign  a 
Poisson  prior  on  the  number  of  errors,  N,  In  nn  order  statistic  model  and  obtain 
a  Poisson  procoss  with  moan  function  u(t)  *  tiF(t),  whore  a  Is  tho  prior  moon.  This 
was  pointed  out  by  Langberg  and  Singpurwiilln  (1981).  A  summary  of  tho  relationships 
between  the  models  discussod  thus  far  appears  In  Figure  1. 

Wo  end  this  section  by  pointing  out  that  n  subclass  of  tho  order  statistic 
processes  is  well  adapted  to  modeling  software  faLluros  (or  more  generally  problems 
In  which  reliability  growth  is  occurring).  l.ittJowood  has  suggostod  that  any  soft¬ 
ware  reliability  model  should  possess  certain  features  among  which  are  the  stochastic 
ordering  and  decreasing  failure  rate  (DFR)  property  of  the  random  variables 
where  Yj  Is  tho  time  between  the  (i-l)-st  and  l-th  failure.  Stochastic  ordering  will 
be  designated  by  s  ,  *  ,  In  nn  order  statistic  modol  Y^Y^...  nre  the  ordorod 
spacings  from  the  distribution  P.  If  F  is  l)FR,  the  Y^s  do  possess  the  desired  prop¬ 
erties  as  the  following  result  shows. 


Theorem  2;  Let  Y^,...,Yn  bo  the  intorurrival  times  of  events  in  an  order  statistic 
process.  If  the  distribution  F  is  DFR  then 

Y.  s  Yj  *  ...  S  Y^ 

1  ST  ST  ST  " 

b)  Yj , . .  ,Y^  are  associated 

c)  is  DFR  i-l,,,.,N. 

Proof ;  Part  (a)  follows  directly  from  Theorem  6.1  in  Barlow  et  al,  (1972). 

Lot  h(t)  bo  the  hazurd  rate  of  the  distribution  F(t),  that  is 
F(t)  “  1  -  exp{-/^h(x)dx},  then  h(t)  it  by  assumption.  An  easy  calculation 
shows  the  conditional  hazard  rate  of  Yj  (Yj, Yj, . .. ..j)  to  be  (N-k+1)  h(t+^"iYi). 
Since  this  decreases  in  Y^,  i«l,...,k-l  it  follows  that  Y.,...iYn  arc  conditionally 

,  I' 

increasing  in  sequence  which  in  turn  implies  the  association  of  Y.j . . . ,  Y.,, 

;  ^  IN 

Yi  is  DFR  since  its  distribution  function  is  a  mixture  of  DFR  distributions, 
llurlow  and  Prosclum  (1975). 


3.  INFERENCE  AND  GOODNESS- OF -FIT 


In  tho  application  of  software  reliability  models  it  has  been  the  practice 
to  use  information  contained  in  the  first  u  failuro  times  XI>...,Xn»  to  estimate 
unknown  parameters.  For  a  majority  of  cases  the  quality  of  estimates  so  obtained 

has  been  far  from  acceptable.  In  tho  present  section  we  study  this  problem  for  the 

'\ 

class  of  order  statistic  models,  and  in  particular  the  model  of  Jolinski  and  Moranda. 
The  parameter  of  paramount  interest  is  N,  tho  initinl  number  of  errors.  For  an  or¬ 
der  statistic  model  X,  <  X«  <...<X„  are  tho  first  n  ordered  random  variables  from  a 

1  Z  fl 

population  of  size  N  with  distribution  function  l'(x)  ■  F(x|j»),  whore  g  is  a  vector 
of  paramotors,  ofton  unknown.  Thus,  estimating  orror  content  is  equivalent  to  esti¬ 
mating  population  size  when  observations  arc  obtained  by  trunentod  sampling.  In 
what  follows  wo  take  l'(x)  ■  1  -  oxp(-Xx)  (Jolinski -Mornndn  model)  although  many  of 
our  commonts  pertain  to  tho  gonoral  case  as  well. 

There  are  several  data  sets  from  completed  software  projects  which  have  been  used 
to  examine  tho  validity  of  tho  Jel Imikl-Mornndn  model.  .Since  tho  softwuro  has  boon 
thoroughly  debugged  N  is  known  as  well  as  the  failuro  times  X^,...^,.  Maximum  like¬ 
lihood  estimates  of  N  are  calculated  from  tho  first  n  <  N  failuro  timos  and  compared 

A 

to  the  true  value.  Typically  It  has  boon  found  that  either  N  grossly  ovorostimntos 

N  (often  N  ■  »)  or  on  tho  other  oxtremo  N  significantly  underestimates  n  (i.e., 

* 

N  *  n  or  n+1).  We  will  show  that  the  formor  behavior  is  to  be  expected,  However,  the 

latter  is  not  and  suggosts  tho  Innpproprlotonoss  of  the  modol. 

For  now,  considor  the  case  where  oach  of  tho  times  X^,  1«l,..,,n  ore  observable. 

If  touting  is  stopped  aftur  a  fixed  time  t(),  oud  in  that  time  u  failures  arc  rocordod, 

tho  likelihood  function  basod  on  X, , . . . , Is 

l  n 

n 

!  (M)  ■  l  ll(N-i+l)Aexp(-M,)  >  oxpl-\tft(N-n) ]  .  (3.1) 

1  ’  0 

If  estimates  are  desired  after  exactly  nQ  errors  have  boon  detected,  the  likelihood 
function  is  obtained  from  (3.1)  by  replacing  n  with  and  with  \  .  Maximum 
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likelihood  estimates  for  both  sampling  schemes  havo  boon  studiod  by  Blumenthal  and 
Morcus  (1975).  The  MLLi  is  finite  if  an  only  if 


i  x.  <  (n+l)t0 
i-1  1  0 


(3.2) 


for  th«  case  of  truncntod  sampling,  and 
n„ 


l  x.  <  C»0+i)x 
i-l  1  0  nc 


(3.3) 


when  sampling  stops  after  failures.  Moro  importantly,  thoir  results  indicate 
that  for  moderately  small  values  of  6  ■  1  -  oxp(-XtQ)  a  fairly  large  positive  bias 


persists  even  when  N  is  finite.  Blumonthal  and  Marcus  also  consider  the  conditional 
MLI2  and  a  class  of  Bnyos  modal  estimates.  On  the  basis  of  second  order  asymptotic 
properties,  one  of  the  hayos  estimates  is  preferred  by  tho  authors. 

Let  us  examine  more  closely' tho  MM2  of  N  when  sampling  stops  aftor  nQ  observa¬ 
tions.  Set  Yj  «  Xj  and  Y^  ■  Xj  -  X^j,  1-2, ...,ny.  Then  the  inequality  in  (3.3) 
is  satisfied  iff  tho  slope  of  the  regression  lino  of  Yj  on  i  is  positive.  Further- 

moro  if  we  let  m  oqunl  this  slope,  then,  conditional  on  .  Y.  ■  X_  ,  N  can  be 

i-i  i 

shown  to  be  a  decreasing  function  of  m.  Thus  if  the  Y^'s  exhibit  marked  reliability 

A 

growth  (m  large)  N  tends  to  be  small.  Under  the  assumption  of  exponential  failure 
timos  ono  would  expoct  positlvo  but  not  largo  vnluos  of  mj  however,  sampling  from  a 
(strictly)  DPR  population  would  account  for  this. 

The  dntn  in  Table  1  is  from  J.  Musa's  "Software  Reliability  DatH",  available 
from  DACS,  Romo  Air  Development  Conter,  NY,  it  contains  tho  ordered  times  betweon 

A 

failuros  Y1,...,Y3g,  N  -  38.  Table  2  shows  tho  value  of  N  computed  from  sample  sizes 

A 

»q  «  5,  10, .. .  ,35,38.  As  can  bo  soon,  N  significantly  iiiulorostiiiiatos  N.  This  phonom 
onon  has  boon  observed  In  other  data  sots  as  well  (Forman  and  Slngpurwulia,  11)77). 


Our  analysis  would  suggest  that  tho  assumption  of  oxponontiallity  (Jelinski-Moranda 
model)  is  not  warranted  for  this  data.  To  verify  this  conjecture  a  test  of  the  hypo 
thesis  H«:  F  is  exponential  against  II.:  F  is  strictly  DPR  was  performed  using  the 


emulative  total  time  on  toet  statistic  of  Barlow  et  al,  (1972).  is  defined  by 

k 

I  (k-j) (k-j+l)Y. 

•i»l  J 

vk  ■  y -  • 

l  (k- j+l)Y , 
j-1  3 

For  the  data  in  Table  1  V.Q  «  13.43.  Using  a  normal  approximation  to  the  distri- 

JO 

bution  of  Vk  one  finds  that 

Pr(Vjg  S  13.43}  -  .002  . 

Since  small  values  of  Vk  favor  DPR  populations  HQ  is  rejected.  For  this  dato  the 
Jolinski-Moranda  model  is  inappropriate.’ 

This  type  of  posterior  analysis  should  ho  useful  in  determining  the  relative 
merit  of  various  order  statistic  modols.  In  asking  whether  the  model  of  Jolinski 
and  Moranda  provides  a  better  fit  than  does  Littlewood's  we  are  in  fact  questioning 
whether  the  variability  in  X^,...,XN  is  bettor  explained  by  an  exponential  distri¬ 
bution  than  a  Pareto  distribution.  Procedures  for  unswering  such  questions  are 
well-known. 


Wo  make  one  final  comment  pertaining  to  tho  analysis  of  grouped  data.  Quite 
often  tho  only  information  available  is  of  tho  form  r,^  errors  detected  in  the  inter¬ 
val  (X^.XJ  l«l,,..,k.  In  the  past  it  was  thought  necessary  to  assume  the 
failure  times  uniformly  distributed  over  the  interval  in  order  to  obtain  estimates 
from  tho  Jelinski-Moranda  model,  With  tho  knowledge  that  wo  arc  obsorving  an  ordor 
statistic  process  it  becomes  evident  that  the  number  of  failures  in  disjoint  inter¬ 
vals  follows  a  multinomial  distribution,  Thus  we  have  the  likelihood, 

r-k  [exp(-XX,  O-oxpf-AXJ]  i 

L(M)  -  - ^ - - - ‘J  expt-A(N-r)Xk1  (3.4) 
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whoro  is  tho  number  of  failures  In  (x(_|,X||  l»l,...,k,  0  ■  X0<...<Xk>  and 
r  ■  Tj  +...+rk.  Sanathanan  (1972)  considorod  estimating  the  site  of  multinomial 
populations  whon  coll  probabilities  aro  differentiable  functions  of  an  unknown 
parameter.  The  interested  reader  should  consult  this  paper  for  details. 

4.  SUMMM1V  AND  CONCLUSIONS 

The  problem  of  modeling  software  failures  is  a  challenging  problem  which  to 
dnto  has  no  satisfactory  solution.  In  an  attempt  to  organiso  tho  litoraturo  wo 
Ituvo  shown  that  several  well-known  models  are  special  eases  of  suporlmposed  dolayod 
transient  renewal  processus.  'In  particular  tho  models  of  Jolinskl  nnd  Morandu  (1972) 
nnd  Mttlowood  (11181)  wore  shown  to  he  moml>ors  of  tho  class  of  order  statistic  pro¬ 
cesses,  which  In  turn  isrolutod,  via  conditioning  arguments,  to  a  subclass  of  the 
Poisson  processus,  namely  those  with  boundod  moon  functions.  More  importantly,  tho 
estimation  problems  oncountored  in  using  these  models  were  put  in  their  proper  per¬ 
spective.  Tho  relevance  of  tho  lUumonthai  and  Marcus  (1975)  results  had  previously 
escaped  tho  attention  of  workors  In  this  field. 

There  is  currently  a  considerable  amount  of  intorost  in  determining  which  model 
works  best,  Wo  have  pointed  out  n  simple  procedure  for  verifying  modol  assumptions 
against  existing  dntn  Hots  ond  comparing  tho  fit  of  cortain  models.  Unfortunately, 
having  the  correct  model  does  not  guarantee  ono  ef  obtaining  reliable  maximum  like¬ 
lihood  ostl mates  of  error  contont.  Since  this  is  the  parnmotor  of  paramount  interest, 
it  is  necessary  to  oxomlno  alternative  forms  of  estimation.  Ungberg  and  Singpur- 
wulln  (1981)  have  addressed  this  problem  with  n  llayosian  approach  which  also  allows 
them  to  unify  some  of  tho  litoraturo  on  softwnro  reliability  modols. 
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METHODOLOGY  FOR  ESTIMATING  MISSION  AVAILABILITY  AND 
RELIABILITY  FOR  A  MULTIMODAL  SYSTEM 


Henry  P,  Bats 

U.S.  Army  Material  Systems  Analysis  Agency 
Aberdeen  Proving  Ground,  Maryland 


1.  INTRODUCTION 

Developmental  Testing  constraints  sometimes  require  that  a 
system  be  tested  sccording  to  a  profile  that  is  different  from  the 
mission  profile  for  which  the  system's  reliability  requirements  were 
specified.  For  example,  s  surface  to  air  missile  system  whose  capa¬ 
bilities  include  movement,  surveillance,  and  target  engagement  might, 
because  of  accelerated  testing  requirements,  be  tested  extensively  in 
the  target  engagement  mode  (in  order  to  assure  that  all  engagement 
performance  requirements  are  mat)  and  only  minimally  in  the  movement, 
and  surveillance  modes  (Figure  1).  However,  in  the  tactical  mission 
profile,  surveillance  functions  might  encompass  the  majority  of  the 
mission  (Figure  2).  It  would  be  incorrect  to  compare  the  system  avail¬ 
ability  and  mean  time  between  failure  (MTiF)  demonstrated  in  the  test 
scenario  to  the  requirements  specified  for  the  tactical  scenario,  This 
is  due  to  the  fact  that  the  engagement  mode  of  operation  ia  more  complex 
and  therefore,  many  more  failures  associated  with  it  would  be  expected, 
Although  the  rate  of  failure  detections  experienced  in  the  engagement 
mode  of  the  test  scenario  would  remain  the  same  as  in  the  tactical  one, 
the  amount  of  time  spent  in  the  engagement  mode  of  the  tactical  scenario 
is  much  less  than  the  test  scenario  which  means  that  a  smaller  number 
of  engagement  mode  failures  should  be  expected  on  a  per  mission  basis, 

In  this  situation,  it  can  be  seen  that  evaluating  the  systom  MTBP  based 
on  the  test  scenario  would  understate  the  MTBF  value.  In  order  to 
determine  if  the  system  meets  its  reliability  specifications,  the  re¬ 
liability  of  the  system  in  the  tactical  mission  must  be  evaluated  from 
data  collected  in  a  test  scenario  which  is  entirely  differont. 

This  report  will  develop  a  methodology  that  can  bo  used  to 
evaluate  a  system  which  ia  operated  in  s  series  of  n  modes  with  the 

i t*'  mode  being  defined  as  having  a  certain  number  of  subsystems  operating 
in  it  and  mode  i  ♦  l  consists  of  mode  i  subsystems  plus  additional  sub¬ 
systems  operating.  That  is,  subsystems  operating  in  mode  i  are  nested 
in  mods  i  ♦  1  (Figure  3) . 

In  addition,  the  corrective  maintenance  time  and  logistics 
delay  time  that  will  be  seen  in  the  field  are  not  always  known  at  the 
time  of  development  testing,  either  because  maintenance  procedures 
are  not  fully  specified  at  that  time  or  for  expediency's  sake  contractor 
personnel  perform  maintenance  normally  done  by  the  soldier.  This  rsport 
allows  for  the  insertion  of  maintsinabllity  parameters  derived  from 
other  sources  i.e.  maintainability  demonstrations,  logistics  simulations, 
etc . 
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DERIVATION  OF  EQUATIONS 


The  first  step  in  deriving  MTBF  and  operational  availability 
estimates  is  to  begin  with  the  basic  definition  of  operational  avail* 
ability  from  which  we  will  derive  the  mission  profile  MTBF.  It  is 
assumed  that  estimates  of  failure  rates  or  WTBFs  for  each  of  the 
operating  modes  are  available.  The  basic  definition  of  operational 
availability  [1]  is  given  by 

a  -  UPTIME  „  . . M 


Let  us  define 


■  time  spent  in  mode  i  as  specified  by  desired  scenario 

■  mean  time  between  failure  detections  of  operating 
mode  i  (derived  from  test  scenario) 

■  mean  downtime  including  logistics  downtime  in 
operating  mode  i  (either  specified  in  tactical 
scenario  or  determined  by  other  means  -  maintainability 
demonstration,  simulation,  etc.)  If  these  individual 
values  cannot  be  determined,  use  MDT  for  all  values 

of  MDTr 

-  Overall  system  mean  downtime  calculated  as 


if  MDT^'s  are  available. 


Otherwise,  use  an  overall  MDT  from  test,  simulation, 
etc. 

Now.  total  time  is  uptime  plus  downtime,  or  uptime  equals 
total  time  minus  downtime.  From  the  definitions  it  may  be  noted  that 
total  mission  time.  T.  is  given  by 


In  determining  downtime,  it  may  be  noted  that 

ki 

MTBT1  ♦  MBT1 

is  the  expected  number  of  failures  in  operating  mode  i  and  multiplying 
this  by  the  expected  downtime  for  mode  i,  MDTs ,  gives  the  expected 
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downtime  in  mode  i.  ThorH'or", 


UPTIME 


[[ki  '(ffrffF"'  *  MDT^j  MDTij 


and  thus,  it  follows  that 


>  ^  ,ki  ♦  mdtJmdti 


It  nay  bo  shown  that  Afl  is  a  weighted  avarsge  of  the  mode 
availabilitios.  On  rearranging  (3),  wo  havo 


ki(MrBFi  4  “Wj-  k4  MDT, 

tfl'8P1  ♦  MbTt  - L 


.  1 1  klMrBpi 
T  J  MTBPJTmotJ' 

/kA/  MTBF,  \ 

•  i  *i  \i 


Now,  wo  may  also  view  A  as 

0 


whoro  MTBP#y>  i,  the  system  MTBF.  Equating  (3)  and  (4)  wo  have 

?n  *  MDTJ 

ffTgP~T  MbT  "  -*■ - -V-  - - J 
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Kcarrunjjing  and  solving  for  system  MTB!;  yields 


MTBF.v.  •  - c - MDT  (5) 

•y*  r  *i 

\  mr^rmq 


3.  EXAMPLE  OF  APPLICATION 

Consider  a  surface  to  air  missile  system  which  is  characterized 
by  three  modes  of  operation  -  travel,  surveillance  and  target  mngagement. 
A  system  MTBP  requirement  of  100  hours  and  an  operational  availability 
requirement  of  0.90  have  bean  set.  The  typical  24  hour  scenario  for 
which  the  requirements  were  set  is  as  follows: 

Time  (hours! 

Travel  1 

Surveillance  21 

Engagement  2 

During  the  test  program  the  following  failure  detection  rates 
were  observed: 


MTBPi 


Travel  1000 

Surveillance  500 

Engagement  50 

Overall  moan  time  to  repair  was  determined  to  be  6  hours  and  mean 
logistics  delay  time  was  found  by  a  logistics  simulation  to  be  14  hours. 
No  other  information  is  available. 

The  question  is  then,  has  the  system  demonstrated  requirements? 
Using  equation  (5)  we  have 


MTBF 


*Y* 


MDT 
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RELIABILITY  PREDICTIONS  FOR  BLACK  HAWK  PRODUCTION  AIRCRAFT 
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ABSTRACT 

f 

In  the  final  stage  of  the  BLACK  HAWK  helicopter  prototype  development,  a 
large  number  of  delayed  fixes  were  proposed  for  the  production  aircraft.  The 
BLACK  HAWK  Project  Manager  requested  that  AMSAA  evaluate  the  Impact  of  these 
delayed  fixes  on  the  reliability  of  the  production  aircraft.  A  methodology 
was  developed  to  predict  the  reliability  of  the  production  aircraft  based  on 
estimates  of  the  effectiveness  of  the  delayed  fixes  and  estimates  of  the  rate 
of  occurrence  of  new  failure  modes. 


1.  INTRODUCTION 

In  December  1976,  Sikorsky  Aircraft  was  awarded  a  production  contract  for 
the  BLACK  HAWK  utility  helicopter  (UH-60A).  However,  further  development  work 
was  needed  on  the  prototype  aircraft  prior  to  production  deliveries.  This 
period  of  advanced  prototype  development  was  called  the  Maturity  Phase.  Dur¬ 
ing  this  period,  a  significant  degradation  In  the  aircraft  system  reliability 
was  observed.  This  degradation  was  attributed  primarily  to  a  more  severe 
contractor  test  environment  and  wear-out  of  some  aircraft  components.  An 
additional  aggravating  factor  was  the  deferral  of  all  reliability  Improvements 
to  the  production  phase.  Consequently,  no  fixes  or  engineering  modifications 
would  be  tested  on  the  prototype  aircraft.  These  circumstances  prevented  the 
BLACK  HAWK  Project  Manager  from  determining  the  progress  of  the  aircraft  to¬ 
ward  meeting  the  established  system  reliability  goal  of  4.0  hours  MTBF.  In 
order  to  determine  the  likelihood  of  meeting  this  goal,  the  Project  Manager 
requested  that  the  Army  Materiel  Systems  Analysis  Activity  (AMSAA)  conduct  an 
evaluation  of  the  effectiveness  of  the  delayed  fixes  and  predict  the  system 
MTBF  of  the  production  aircraft.  This  evaluation  would  allow  the  Project 
Manager  to  make  timely  managerial  and  technical  decisions  In  order  to  correct 
problem  areas  prior  to  production. 

2.  METHODOLOGY 

2.1  Basis  for  Prediction.  The  prediction  cf  the  reliability  of  the 
production  BLACK  HAWK  was  based  on  analysis  of  existing  Maturity  Phase  fail¬ 
ure  modes  In  light  of  proposed  engineering  fixes,  and  the  merging  of  this 
analysis  with  an  estimate  of  the  rate  of  occurrence  of  new  failure  modes. 
Existing  Maturity  Phase  failure  modes  were  also  evaluated  to  Identify  any 
modes  which  exhibited  wear-out  characteristics  or  which  were  discovered  during 
a  modification  or  special  test  procedure  peculiar  to  the  contractor  test 
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environment.  These  type  of  failure  modes  were  considered  to  be  "unlikely 
occurrences"  on  new  production  aircraft. 

2.2  Analysis  of  Existing  Failure  Modes.  The  analysis  of  existing  failure 
modes  was  based  on  failures  occurring  during  467  flight  hours  of  Maturity 
Phase  testing.  A  total  of  273  failures  were  charged  to  Sikorsky  during  this 
457  flight  hour  period.  An  Army  team  of  engineering  personnel  was  organized 
by  AMSAA  to  estimate  the  effectiveness  of  the  contractor  fixes  for  these 
failures.  The  effectiveness  of  a  modification  on  any  given  failure  mode  was 
evaluated  by  means  of  the  average  effectiveness  factor  (k  factor)  assigned  to 
that  mode  by  the  Army  team.  For  example*  a  k  factor  of  .60  assigned  to  a  fix 
for  a  particular  failure  mode  over  a  certain  time  Interval  would  Indicate 
that  after  the  fix  Is  Incorporated,  60*  of  the  number  of  failures  of  that 
particular  mode  over  the  same  time  Interval  would  not  be  expected  to  occur; 
that  Is,  only  40*  of  the  number  of  failures  would  be  expected  to  occur  over 
the  same  time  Interval  If  the  fix  were  Incorporated. 

Thus,  If  N  represents  the  number  of  failures  of  a  particular  mode  occurr¬ 
ing  over  a  certain  time  period,  and  E(N )  represents  the  expected  number  of 
failures  over  the  same  time  period  after  the  fix  with  effectiveness  factor  k 
Is  Incorporated,  then 

E(N)  «  N(l-k). 

Fixes  were  proposed  by  Sikorsky  for  243  of  the  273  failures  during  the 
Maturity  Phase.  Table  1  provides  a  breakdown  of  the  243  failures  by  major 
subsystems  and  the  expected  number  of  failures  In  each  subsystem  after  apply¬ 
ing  to  each  failure  In  the  subsystem  Its  respective  k  factor  as  determined  by 
the  Army  team.  After  applying  the  k  factors  to  the  243  failures  for  which 
fixes  were  proposed,  the  expected  number  of  failures  based  on  457  flight  hours 
was  reduced  from  243  to  91.85. 
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TABLE  1.  FAILURES  FOR  WHICH  FIXES  WERE  PROPOSED 


& 

SUBSYSTEM 

NUMBER  OF 
FAILURES 

EXPECTED 
NUMBER  OF 
FAILURES* 

EXPECTED  FAILAURE 
RATE  (BASED  ON 

457  HOURS 1 

& 

fc, 

Rotor 

1  42 

11,61 

0.025 

jr  ■ 

P- 

Transmission 

6 

2.08  ' 

0.005 

k- 

fe 

Propulsion  (GFE) 

13 

5.55 

0.012 

I 

Propulsion  (CFE) 

25 

8.75 

0.019 

i 

Electrical 

27 

12.11 

0:026 

k 

Avionics 

i 

33 

20.64 

0.045 

V 

r 

Airframe 

32 

11.65 

0.025 

V  L. . 

k" 

;;  (i.  _ 

‘i  l‘v'  ' 

Electronic 

Controls 

, 

15 

2.56 

0.006 

Hydraulic/ 

Flight  Controls 

i 

50 

16.80 

0.037 

1' 

[i\ 

TOTAL 

/ 

1 

243 

91.85 

0.200 

♦Obtained  by  applying  to  each  failure  the  average  k  factor  assigned  by  the 
Government  Evaluation  Team 
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It  was  anticipated  that  prior  to  production  deliveries,  fixes  would  be 
Installed  for  the  remaining  30  failures  out  of  the  total  of  273  failures. 

The  effectiveness  of  these  fixes  was  estimated  by  categorizing  the  30  failures 
by  subsystem,  and  applying  the  average  subsystem  k  factor  derived  from  the 
total  243  failures  reviewed  to  the  respective  numbir  of  failures  In  each  sub¬ 
system.  Table  2  provides  the  breakdown  of  the  30  failures  by  subsystem  and 
the  expected  number  of  failures  after  application  of  the  k  factors  discussed 
above.  The  expected  number  of  failures  was  thus  reduced  from  30  to  11.6. 

The  273  failures  which  were  charged  to  Sikorsky  during  the  457  flight  hour 
period  were  evaluated  to  determine  those  modes  which  exhibited  wear-out  char¬ 
acteristics  or  which  were  discovered  during  a  modification,  special  Inspection, 
or  experimental  procedure  peculiar  to  the  contractor  testing.  These  modes 
were  not  considered  likely  to  occur  on  production  aircraft,  The  Army  team 
Identified  46  unlikely  occurrences  among  the  243  failures  with  fixes.  An 
AMSAA  Independent  analysis  Identified  an  additional  16  unlikely  occurrences 
among  the  243  failures  with  fixes  and  the  30  failures  without  fixes.  The 
AMSAA  analysis  thus  considered  62  failures  as  unlikely  occurrences  among  the 
total  273  failures.  Fifty-eight  of  the  62  unlikely  occurrences  were  dis¬ 
covered  among  the  243  failures  with  fixes,  yielding  a  total  of  185  failures 
with  fixes  and  purged  of  unlikely  occurrences.  Four  of  the  62  unlikely 
occurrences  were  discovered  among  the  30  failures  without  fixes,  leaving  26 
failures  without  fixes  and  purged  of  unlikely  occurrences, 

Table  3  provides  a  breadown  by  subsystem  of  the  185  failures  with  fixes 
and  purged  of  unlikely  occurrences,  and  the  number  of  these  expected  to  occur 
after  applying  the  k  factors  of  the  Army  team.  The'  expected  number  of  failures 
after  the  k  factors  are  applied  was  reduced  from  185  to  70.2. 
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TABLE  2.  FAILURES  WITHOUT  FIXES 


SUBSYSTEM 

NUMBER  OF 
FAILURES 

SUBSYSTEM 

K  FACTOR 

EXPECTED 
NUMBER  OF 
FAILURES* 

EXPECTED  FAILURE 
RATE  (BASED  ON 
457  HOURS) 

Rotor 

2 

.723 

.55 

0.001 

Transmission 

3 

.653 

1.04 

0.002 

Prooulslon  (QFE) 

0 

.573 

0.00 

0.000 

Propulsion  (CFE) 

3 

.650 

1.05 

0.002 

Electrical 

4 

.551 

1.80 

0.004 

Avionics 

4 

.374 

2.50 

0,005 

Alrframa 

2 

.635 

1.27 

0.003 

Elactronlc 

Controls 

4 

.829 

0.68 

0.001 

Hydraulics/ 

Flight  Controls 

8 

.662 

2.70 

0.006 

TOTAL 

30 

11.59 

0.024 

*Obta1nad  by  applying  tha  subsystam  avafaga  k  factor  to  tha  fallurai  In  that 
subsystem 
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TABLE  3.  FAILURES  FOR  WHICH  FIXES  WERE  PROVIDED  (PURGED  OF  UNLIKELY  OCCURRENCES) 

EXPECTED  FAILURE 

NUMBER  OF  EXPECTED  NUMBER  RATE  (BASED  ON 

SUBSYSTEM  FAILURES  OF  FAILURES*  457  HOURS) 

Rotor 

21 

5.80 

0.013 

Transmission 

3 

0.97 

0.002 

Propulsion  (GFE) 

8 

3.92 

0.009 

Propulsion  (CFE) 

20 

6.32 

0.014 

Elactrlcal 

24 

11.77 

0.026 

Avionics 

29 

18.36 

0.040 

A1 rf rama 

23 

6.85 

0.015 

Elactronlc 

Controls 

IB 

2.56 

0.006 

Hydraulics/ 

Flight  Controls 

42 

13.63 

0.030 

TOTAL 

IBS 

70.18 

0.155 

♦Obtained  by  applying  to  aoch  fallura  tha  avaraga  k  factor  asslgnad  by  the 
Governmant  Evaluation  Taam  (Appandlx  A] 
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Table  4  provides  the  breakdown  of  the  26  failures  without  fixes  and  purged 
of  unlikely  occurrences  and  the  expected  number  of  failures  after  application 
of  the  k  factors  discussed  above.  The  expected  number  of  failures  was  thus 
reduced  from  26  to  10.3. 

Table  5  combines  the  results  of  Table  3  and  Table  4  to  give  the  total  ex¬ 
pected  number  of  failures  (purged  of  unlikely  occurrences)  after  proposed 
corrective  actions  (Table  3)  and  after  anticipated  corrective  actions  (Table 
4)  are  Incorporated.  A  total  of  80.5  expected  failures  (purged  of  unlikely 
occurrences)  was  thus  obtained,  yielding  a  failure  rate  of  0.178  based  on  457 
flight  hours. 

2.3  Analysis  of  New  Failure  Modes.  It  was  expected  that  the  production 
aircraft  would  experience  new  failure  modes  which  had  not  been  seen  on  pro¬ 
totype  aircraft,  An  estimate  of  the  rate  of  occurrence  of  new  failure  modes 
on  production  aircraft  was  obtained  by  considering  the  rate  of  occurrence  of 
new  failure  modes  through  all  Basic  Engineering  Development  (BED)  Phase  and 
Maturity  Phase  flight  testing. 

Figure  1  presents  a  log-log  plot  of  the  cumulative  rate  of  occurrence  of 

new  failure  modes  versus  the  cumulative  test  time  during  the  BED  Phase  and 

Maturity  Phase  flight  testing.  The  linear  fit  of  the  data  on  the  log-log 

plot  Indicates  that  the  occurrence  of  new  failure  modes  follows  a  non-homogen- 

0  1 

eous  Poisson  process  with  Intensity  function  given  by  r(t)  ■  XBt  (Reference 
1).  The  function  r(t)  represents  the  Instantaneous  rate  of  occurrence  of  fail¬ 
ure  modes.  This  function  Is  shown  In  Figure  2  with  estimates  of  \  and  B  ob¬ 
tained  from  the  AMSAA  Reliability  Growth  Model  (Reference  1).  The  expected 
rate  of  occurrence  of  new  failure  modes  on  the  production  aircraft  was  esti¬ 
mated  as  follows: 

If  E(N(t))  represents  the  expected  number  of  new  failure  modes  In  time  t, 


TABLE  5.  TOTAL  FAILURES  (PURGED  OF  UNLIKELY  OCCURRENCES) 


SUBSYSTEM 

NUMBER  OF 
FAILURES 

EXPECTED 
NUMBER  OF 
FAILURES* 

EXPECTED  FAILURE 
RATE  (BASED  ON 
457  HOURS) 

Rotor 

21 

5.80 

0.013 

Transmission 

6 

2.01 

0.004 

Propulsion  (GFE) 

8 

3.92 

0.009 

Propulsion  (CFE) 

21 

6.67 

0.015 

Electrical 

28 

13.57 

0.030 

Avionics 

33 

20.86 

0.046 

Airframe 

25 

8.12 

0.018 

Electronic 

Controls 

19 

3.24 

0.007 

Hydraulics/ 

Flight  Controls 

50 

16.33 

0.036 

TOTAL 

211 

80.52 

0.178 

♦Obtained  by  adding  tha  number  of  failures  In  each  subsystem  from  Table  4 
and  Table  5 
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FIGURE  1 


JGHT  HOURS 


B~1  B 

then  t(N(t) )  ■  j  xBt  dt  «  \t  . 

*  o 

As  depicted  In  Figure  2,  production  testing  on  the  BLACK  HAWK  began  after 
approximately  2500  flight  hours  of  testing  during  the  Bas<c  Engineering  Develop¬ 
ment  (BED)  Phase  and  the  Maturity  Phase.  This  testing  on  f1r?t  production  year 
aircraft  was  expected  to  accumulate  1500  flight  hours  which  would  extend  the 
total  flight  time  on  prototype  and  production  aircraft  to  4000  flight  hours. 

The  number  of  new  failure  modes  occurring  on  production  aircraft  would  then 
be  those  occurring  between  2500  and  4000  flight  hours  on  Figure  2.  The  ex¬ 
pected  number  of  new  failure  modes  between  2500  and  4000  flight  hours  was  thus 
estimated  by  E(4000)  -  E(2500)  «  x(4000)8  -  x(?500)8  ■  124. 

This  value  was  obtained  with  the  estimates  of  \  and  3  shown  In  Figure  2. 

The  expected  rate  of  occurrence  of  new  failure  modes  was  then  estimated  to  be 
124/1500  -  0.08. 

2.4  Prediction  of  System  MTBF.  The  prediction  of  system  MTBF  was  obtain¬ 
ed  by  adding  the  rate  of  occurrence  of  new  failure  moues  tc  the  failure  rate 
of  Table  5  which  represents  tne  rate  for  existing  failure  modes  purged  of  un¬ 
likely  occurrences  and  adjusted  for  contractor  fixes.  The  calculation  Is 
given  below: 

0.08  (unseen  failure  mode  rate)  +  0.178  (Table  5)  ■  0  258  failures  per  hour. 

The  failure  rate  of  0.258  corresponds  to  a  system  MTBF  of  3.9  hours. 

3.  CONCLUSIONS 

The  system  MTBF  prediction  of  3.9  hours  Indicated  to  the  Project  Manager 
that  no  drastic  measures  In  terms  of  program  cost  or  testing  would  be  required 
to  meet  the  MTBF  goal  of  4.0  hours.  At  the  same  time,  however,  It  was  appar¬ 
ent  that  the  contractor  could  not  afford  tc  reduce  his  efforts  In  improving 
reliability. 


137 


The  production  SLACK  HAWK  aircraft  was  subsequently  delivered  to  the  Army 
and  demonstrated  an  MTBF  of  3.7  hours  as  reported  by  the  US  Army  Aviation 
Beard  during  Initial  production  testing.  This  value  compares  favorably  with 
the  prediction  of  3.9  hours  MTBF  developed  by  the  methodology  In  this  paper. 
However,  the  MTBF  of  3.7  reported  by  the  Aviation  Board  did  not  Include  a 
large  number  of  deferred  maintenance  actions.  These  deferred  actions  were 
largely  quality  control  defects  peculiar  to  Initial  production  deliveries. 

If  these  defects  are  Included,  the  MTBF  of  the  early  production  aircraft  Is 
reduced  from  3.7  hours  to  2.9  hours.  In  future  applications  of  the  meth¬ 
odology  In  this  paper,  an  adjustment  will  be  required  to  account  for  large 
numbers  of  production  line  discrepancies. 
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ABSTRACT .  Two-phase  sampling  Is  a  relatively  unknown  technique 
which,  In  military  applications,  could  save  money  In  several  areas  of 
measurement.  By  first  classifying  the  population  through  the  use  of  an 
Inexpensive  method,  one  reduces  the  number  of  expensive  measurements 
which  would  be  required  If  traditional  one-phase  sampling  were  used. 
Two-phase  sampling  achieves  this  by  better  utilizing  Information  gained 
by  classification  than  does  traditional  stratified  sampling. 

I.  Introduction.  The  purpose  of  this  paper  Is  to  Inform  a  larger 
portion  of  the  military  analytic  community  about  the  benefits  and  tech¬ 
niques  of  two-phase  sampling.  In  these  times  of  reduced  budgets  and  In¬ 
creased  emphasis  on  efficiency,  any  technique  which  can  reduce  the  total 
cost  of  an  experiment  or  measurement  should  be  considered  as  an  alterna¬ 
tive  approach  to  the  problem. 

The  first  part  of  the  paper  describes  several  military  application 
areas  In  which  two-phase  sampling  appears  to  be  feasible  and  beneficial. 
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The  second  part  familiarizes  the  reader  with  a  brief  description  of  the 
basic  theory  behind  two-phase  sampling.  The  third  part  gives  the  gen¬ 
eral  problem  formulation  and  solution,  along  with  a  computational  ex- 
ample. 

II.  Possible  Areas  of  Application.  Many  military  studies  require 
descriptive  statistics  on  large  populations  for  which  comprehensive  and 
exhaustive  measurement  Is  either  Infeasible  or  too  costly.  Sampling  Is 
used  to  extrapolate  Information  about  a  set  by  measuring  one  or  more  at¬ 
tributes  of  a  subset  selected  at  random  from  this  set.  One  can  either 
choose  the  number  of  samples  given  a  desired  level  of  confidence,  or  one 
can  obtain  a  confidence  level  given  a  fixed  number  of  samples  or  budget 
limitation.  In  some  cases,  however,  the  rosts  associated  with  taking 
the  number  of  samples  required  for  a  glv.en  confidence  level  are  prohibi¬ 
tive  or  unnecessarily  expensive.  Two-phase  sampling  may  allow  the  ex¬ 
perimenter  to  obtain  the  desired  level  of  accuracy  within  specified  bud¬ 
get  constraints. 

There  are  several  potential  areas  of  application,  both  military  and 
nonmilitary,  In  which  two-phase  sampling  may  be  better  than  single-phase 
sampling.  One  situation  Involves  the  destructive/ nondestructive  methods 
of  testing  the  components  of  a  system.  Destructive  testing  Is  not  de¬ 
sirable  when  the  component  Is  expensive.  If  non-destructive  testing  Is 
applied  Instead,  the  Individual  tests  are  less  costly,  but,  since  a  much 
larger  number  of  them  might  be  required,  total  costs  may  be  no  less  ex¬ 
pensive.  Successive  use  of  both  nondestructive  (Inexpensive)  and  de¬ 
structive  (expensive)  measurements  may  provide  the  desired  accuracy  with 
less  total  cost  than  only  a  single-phase  sample. 
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Another  potential  area  of  military  applications  Is  In  Inventory  man¬ 
agement.  The  Army  maintains  large  numbers  of  Items  on  stock  which  are 
Inventoried  annually.  Usually,  this  Is  done  on  a  10  percent  per  month 
Inventory  cycle  so  that  an  annual  Inventory  Is  completed  by  the  end  of 
the  year;  however,  complete  physical  count  Inventories  are  expensive  and 
time-consuming.  Two-phase  sampling  may  help  In  this  area  as  well.  The 
stock  may  be  partitioned  Into  high  cost  and  low  cost  Items.  The  high 
cost  Items  will  be  completely  Inventoried  by  a  physical  count  and  the 
low  cost  Items  will  be  sampled.  (One  does  not  wish  to  have  to  count 
every  nail,  screw,  or  washer,  although  It  has  been  attempted!  Some  In¬ 
ventory  operations  base  a  count  on  the  weighing  of  many  small  homogene¬ 
ous  Items.)  Information  about  the  state  of  the  complete  stock  may  be 
obtained  by  this  method. 

Another  way  of  Improving  Inventory  control  Is  to  first  segregate  the 
Items  to  be  counted  according  to  demand  records.  The  first  phase  of 
sampling  is  the  sorting  of  the  potential  Inventory  Into  categories  com¬ 
prising  the  most  popular  demand  Items.  The  second  Is  the  physical 
counting  of  those  high  demand  Items.  The  two-phase  sampling  technique 
will  show  how  to  do  this  at  least  cost. 

A  third  area  of  military  application  Is  In  personnel  studies.  The 
A.my  maintains  large  data  bases  describing  various  subsets  of  the  mili¬ 
tary  population  relevant  to  military  performance.  (For  example,  the 
proportion  of  now  recruits  who  went  AWOL  after  receiving  a  monetary  bo¬ 


nus  was  found  to  be  quite  high  at  one  time.)  Measuring  a  certain  attri¬ 
bute  of  the  population  by  personal  Interviews,  or  even  questionnaires, 
can  be  both  time-consuming  and  expensive.  However,  use  of  a  first  phase 
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to  subdivide  the  population  before  taking  the  second,  more  expensive  set 
of  measurements  can  give  the  same  amount  of  accuracy  at  less  cost. 

Two-phase  sampling  Is  not  the  same  thing  as  stratified  random  sam¬ 
pling.  Most  questionnaires  are  based  upon  stratified  random  sampling, 
which  simply  samples  a  random  group  from  the  stratified  categories  de¬ 
fined  by  the  experimenter.  However,  If  the  data  defining  the  strata  are 
old  or  not  currently  accurate,  the  results  of  the  estimation  will  be 
biased.1  This  Is  because  two-phase  sampling  estimates  the  strata 
weights  In  the  first  phase,  which  provides  more  current  estimates  than 
In  stratified  random  sampling. 

III.  A  Brief  Description  of  the  Basic  Theory.  The  classic  presen¬ 
tation  of  two-phase  sampling  was  given  by  Cochran  In  his  book,  Sampling 
Techniques.  The  total  variance  of  the  unbiased  estimator  for  the 
stratified  mean  (ylt)  of  the  variable  y  In  two-phase  sampling  Is  given 
by  the  equation: 


u 


<-  - 1) 


Cl] 


where:  S2  is  the  variance  of  the  total  population, 

S^j  is  the  variance  of  each  stratum  (h>l,...,L), 

wh  is  the  weight  given  to  each  stratum  of  the  population  where 


Wh  -  Nh/N, 

N  Is  the  total  population, 

ty,1s  the  subpopulatlon  of  stratum  h, 

n'  Is  the  size  of  the  first  sample, 

n't,1s  the  number  of  stratum  h  In  the  first  sample, 

Vs  the  number  of  stratum  h  In  the  second  sample, 
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Vj,  Is  the  proportion  of  the  second  sample  taken  from  the  popula¬ 
tion  given  by  the  first  sample  In  that  stratum,  or 

nh  "  vhnfc‘  CZ] 

The  assumptions  required  for  this  to  be  true  are: 

(1)  The  first  sample  must  be  random. 

(2)  The  second  sample  Is  a  random  subsample  of  the  first  sam¬ 
pling. 

(3)  The  first  sample  Is  large  enough  so  that  the  estimated 
weights  (Wh)  are  all  nonzero. 

(4)  'Every  proportion  found  In  the  optimal  solution  Is  less  than 
the  total  number  chosen  In  that  stratum  In  the  first  sample, 
l.e.  n^  is  greater  than  n^. 

The  purpose  of  the  first  sampling  Is  to  determine  the  strata 
weights.  The  purpose  of  the  second  sample  Is  to  estimate  the  strata 
means  so  that  the  population  mean  may  be  estimated  In  an  unbiased  way. 
Stratified  random  sampling  may  give  biased  estimators,  whereas  two-phase 
sampling  will  not.2 

Equation  [1]  may  be  rewritten  as; 

f  m 

Notice  that  If  all  of  the  variances  of  each  stratum  are  Identical, 
then  the  first  term  becomes  zero  and  we  are  back  to  a  single-phase  sam¬ 
pling  problem.  Two-phase  sampling  seems  to  work  the  best  when  a  small 
percentage  of  the  population  has  high  variance  while  the  rest  of  the 
population  has  low  variance.  This  also  shows  how  two-phase  sampling  has 
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a  slight  edge  over  stratified  sampling.  Notice  that  the  variance  terms 
on  the  right-hand  side  of  equation  [3]  are  larger  than  in  the  single 
sample  case,  which  Includes  only  Var(j?st).  Also,  In  each  term  on  the 
left-hand  side,  the  numerator  Is  smaller  than  In  the  single  sample  case. 
Of  course,  the  sum  of  the  terms  on  the  left-hand  side  must  be  less  than 
the  right-hand  side  when  the  confidence  Interval  Is  fixed.  In  the  case 
where  the  optimal  number  of  samples  Is  small  and  the  number  of  strata  Is 
large,  two-phase  sampling  may  do  worse  than  single-phase  sampllngi  how¬ 
ever,  such  conditions  are  rare. 

IV.  General  Problem  Formulation  and  Solution.  In  general ,  the  ex¬ 
perimenter  will  wjsh  to  determine  how  many  samples  should  be  taken  In 
each  phase  and  1n|each  stratum.  Even  though  the  problem  formulation  Is 
nonlinear,  there  Is  a  relatively  simple  method  of  solution.  The  general 
formulation  Is* 

*j  ■  v»,t>  ■  £  *U*i  M 

where  - 

a1j  "  w1s1j  C5] 

and 

x1  -  (l/ni  -  1/Nj).  [6] 


The  objective  Is  to  minimize  K,  where  c^  Is  cost  of  measurement  n^ ,  or 


Min  K 


ci"i 


subject  to: 


C7] 


a1j*1  *  vj 


[8] 
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and 


0  i  x,  i  (1  -  1/N^j 


[9] 


where  1“l,...,p,  where  p  Is  the  number  of  characteristics  addressed 
In  the  problem. 

There  are  two  ways  to  solve  this  formulation  with  a  linear  objective 
function  and  nonlinear  constraints.  One  uses  Lagranglan  multipliers  and 
the  other  uses  geometric  programing.  A  third  solution  uses  a  transfor¬ 
mation  which  makes  the  objective  function  nonlinear  and  the  constraints 
linear,  as  shown  by  Kokan  and  Khan  (1967).  All  methods  give  the  same 
solution, 

*1j  “  *1  j ^ *1  i41  J j ))  CIO] 

where  _L 

KJ"VjL«1j/Nl’  dl] 

The  procedure  for  finding  the  solution  to  a  given  problem  Is  to 
first  find  a  constraint  which  Is  not  dominated  by  any  other  constraint, 
and  then  calculate  the  solution  given  by  equation  [10].  If  this  solu¬ 
tion  satisfies  all  constraints,  the  optimal  solution  has  been  found. 

The  following  example  should  help  explain  the  procedure. 

A  problem  of  estimating  plant  biomass  In  Range  Science  has  been  de¬ 
fined  as  a  two-phase  sampling  problem.  There  are  three  strata  of  plants 
to  be  measured  at  minimum  cost  subject  to  constraints  on  total  variance. 
In  the  constraints  given  below,  the  numbers  for  the  a^  have  been  com¬ 
bined  Into  one  coefficient,  and  the  right-hand  side  has  been  divided 
through.  The  resulting  formulation  Is: 


Minimize  .5n^  +  5n2 


C12] 


such  that 

200/nj  ♦  12/n2  i  1 
175/nj  +  40/n2  i  1 
160/nj  +  30/ng  i  1 
170/nj  +  20/n2  i  1 
175/rij  +  25/r»2  i  1 


where  n^  Is  the  number  of  samples  of  phase  1  to  be  drawn.  We  can 
see  from  the  first  constraint  that  a  lower  bound  on  nj  and  n2  Is  re¬ 
quired  to  be  n2  *  12  and  n^  >  200  for  the  first  constraint  to  be  feasi¬ 
ble.  Similarly,  for  constraint  two,  nt  *  175  and  n2  i  40.  For  the  rest 
of  the  constraints  we  have  nj  1  150  and  n2  i  30,  nj  i  170  and  n2  >  20, 
and  nj  >  175  and  n2  i  25.  But  notice  that  the  last  three  constraints 
are  totally  dominated  by  the  second  constraint.  Therefore,  the  last 
three  constraints  can  be  Ignored  since  they  do  not  Influence  the  problem 
In  the  presence  of  constraint  two.  This  brief  anaysls  gives  us  an  Imme¬ 
diate  lower  bound  on  the  minimum  cost  at  300. 

When  we  try  our  solution  from  equation  [10],  nj  «  354.92  and  n2  * 
27.49  If  we  chose  the  first  constraint  as  being  active.  However,  this 
violates  the  second  constraint.  If  we  choose  the  second  constraint  as 
being  active,  then  the  optimal  solution  Is  obtained:  nj  ■  439.58,  n2  ■ 
66.46,  and  minimum  cost  Is  552.08.  Notice  that  since  the  solutions  are 
noninteger,  one  must  round  up. 


. . -fai-yiri»iMiii*idii*rti>«wm*nw^Mi« 


V.  Conclusions.  The  two-phase  sampling  technique  appears  to  be  ap¬ 
plicable  to  a  large  number  of  military  problems.  When  applicable,  It 
has  potential  for  cost-saving  relative  to  traditional  single-phase  sam¬ 
pling.  Although  not  as  easily  applied  as  stratified  random  sampling, 
two-phase  sampling  usually  will  do  better  especially  when  the  data  are 
not  up-to-date.  The  general  solution  shown  In  this  paper  can  be  applied 
with  the  use  of  a  hand  calculator.  Two-phase  sampling  Is  not  applicable 
when  no  data  or  a  priori  assumptions  about  the  variances  of  the  strata 
are  known  or  when  they  are  expansive  to  obtain. 

Footnotes 

1)  Cochran,  Samuil nq  Techniques,  pp.  117-119. 

2)  Ibid. 
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DESIGN  OF  A  MULTIPLE  SAMPLE  WESTENBEKG  TYPE  TEST  FOK  SMALL  SAMPLE  SIZES 


James  R.  Kneub,  Jr, 

US  Am\y  Logistics  Center 
Fort  Lee,  Virginia 


ABSTRACT ,  This  paper  discusses  an  experimental  design  which  is  bated  on  a 
multiple  sample  extension  of  Westenberg's  Interquartile  Range  Test  (random 
effects)  and  the  Westenberg-Mood  Median  Test  (fixed  effects).  Both  type  1  and 
type  II  error  probabilities  have  been  computerized  for  small  sample  sizes  using 
an  exact  test  which  Is  a  multiple  sample  extension  of  a  two-tailed  Fisher  Exact 
Test.  Unlike  other  distribution  free  analysis  of  variance  tests  such  as 
Kruskal-Wallls  or  Friedman,  random  effects  may  be  Investigated,  and  practical 
type  II  error  analyses  are  available.  When  sample  sizes  are  large,  the 
chi-square  distribution  will  provide  a  reasonable  approximation;  however,  when 
sample  sizes  are  small  this  test  Is  needed. 

Current  plant  are  to  use  this  test  In  coordination  with  other  statistical 
methods  to  analyze  data  from  a  study  being  conducted  concerning  manpower 
requirements  In  the  various  types  of  units  found  In  the  US  Army.  This  will  be 
discussed  briefly  In  this  paper  also. 


INTRODUCTION 

This  paper  Is  based  upon  a  part  of  Annex  B  to  reference  2,  which  Is  a  study 
plan  for  gathering  data  on  the  amount  of  time  expected  to  be  spent  by  soldiers, 
particularly  In  a  wartime  environment,  on  duties  other  than  their  primary 
Military  Occupational  Specialty  (MOS)  duties.  An  earlier  study,  reference  4, 
indicated  that,  for  example,  median  times  to  be  spent  on  Kitchen  Police.  (KP) 
activities  were  expected  to  be  the  same  when  comparing  questionnaire  results 
from  e  number  of  US  Army  posts.  (The  Kruskal-Wallls  test  was  used  as  the 
sample  distributions  were  decidedly  non-normal.)  However,  a  cursory  study  of 
the  data  Indicated  strongly  that  the  variances  were  quite  different.  One 
explanation  would  be  that  various  types  of  Army  units  stationed  at  these  posts 
could  have  Influenced  the  data  In  such  a  way.  In  the  current  study,  reference 
2,  units  are  broken  out  by  category  (combat,  combat  support,  combat  service 
support)  and  logical  region  (division,  corps  and  echelons  above  corps),  and  In, 
some  cases  by  unit  type  (e.q.,  transportation,  chemical,  etc.).  In  reference 
2,  Annex  B  suggests  a  number  of  analytical  methods  for  examining  the  results  of 
this  study  as  designed  In  that  study  plan.  For  comparing  Units  In  three 
logical  regions  (LRs)  and  three  categories  (CATs)  for  both  fixed  and  random 
effects  when  the  assumption  of  normality  may  not  be  reasonable,  the  methods  of 
this  paper  have  been  developed.  The  Statistical  Package  for  the  Social 
Sciences  (SPSS)  will  be  used  for  much  of  the  analysis,  but  the  computer 
programs  for  the  hypothesis  tests  to  be  given  here  are  given  as  an  appendix  to 
this  paper  since  they  ere,  of  course,  not  found  In  the  SPSS. 
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NEW  STATISTICAL  HYPOTHESIS  TEST 
FOR  DISTRIBUTION-FREE  ANOVA 


This  test  Is  basically  a  multiple  sample  extension  of  the  Westenberg 
Interquartile  Range  Test  and  the  Westanberg-Mood  Median  Test.  As  such,  the 
underlying  statistics  are  from  Fisher's  Exact  Test  for  proportions.  At  the 
foundation  of  Fisher's  Exact  Test  Is  the  following  mathematical  expression: 

C(A-rB)!r1B(l-r1)A(C+D)!r2D<l-r2)C]/[A!B!C!D!] 


[(A+B+C+D) ! r0B+D( l-rQ)A+C]/[(B+D) ! (A+C) ! ] 

This  Is  the  probability  of  having  A  "failures"  In  one  sample  when  the 
probability  of  "success"  Is  r^,  and  C  "failures"  In  the  other  sample  with  a 

probability  of  "success"  r2,  normalized  to  having  a  total  of  A+C  "failures"  In 

the  combined  sample  with  overall  probability  of  "success"  rQ.  ("Success"  and 

"failure"  muot  be  defined  In  each  application.  These  words  are  used  more 
accurately  when  Fisher's  Exact  Test  Is  applied  In  the  comparison  of  system 
reliabilities,  however,  when  used  to  judge,  for  example,  whether  two  (or  more) 
samples  appear  to  come  from  populations  with  the  same  Interquartile  range, 
"success"  could  denote  an  observed  value  within  the  Interquartile  range  of  the 
combined  sample,  or  vice  versa.) 

When  determining  the  probability  of  a  type  I  error,  since  the  null 
hypothesis  Is  that  of  equal  probabilities,  r^,  rg  and  rQ  are  set  equal  to  each 

other,  and  the  above  expression  reduces  to: 

This  Is  the  probability  of  obtaining  the  event  observed  given  that  the  null 
hypothesis  Is  true.  It  Is  found  In  reference  6  and  In  other  forms  In  reference 
1  and  other  sources.  In  order  to  determine  the  probability  of  a  type  I  error, 
this  case  and  all  more  extreme  cases  must  be  analyzed  and  their  probabilities 
added.  [This  Is  a  one-tailed  test,  however,  the  multiple  sample  version  (l.e., 
more  than  two  samples),  Is  an  extended  two-tailed  test.] 

The  probability  of  a  type  II  error  was  derived  for  the  Fisher  Exact  Test 
and  tabulated  In  a  study  at  White  Sands  Missile  Range,  to  become  part  of  the 
material  held  at  the  Defense  Documentation  Center,  under  the  title  "Missile 
Round  Sample  Size  Considerations  for  Test  Planning  and  Reporting,"  which  Is 
reference  3.  Under  the  alternative  hypothesis  of  unequal  r^,  r2  and  rQ,  the 

probability  of  the  event  observed  becomes: 

(A+B)!r1B(l-r1)A(C+D)lr2D(1.r2)C 
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where  p=P[total  of  A+C  failures  In  the  combined  sample] 


AtC/A+BWi  „  vX  A+B-x,  C+D  wD-A+xn 
xi0(  x  )0  ->*1)  »*!  'A+C-x'r2 


r2> 


A+C-x 


(When  B  *  C  and  0  ^  A,  otherwise,  not  all  term  will  be  present) 

To  obtain  the  probability  of  a  type  II  error,  the  case  observed  and  all  less 
extreme  cases  must  be  analyzed  and  their  probabilities  added. 

The  general  expression  for  a  multiple  sample  version  of  this  test  Is 

*  V'nS  -  rj k3HtJiVtCkliBt]l> 

<jhWrOkt1  ■  r0>  +  Bk»'> 

Where, 

A^»The  number  of  "failures"  In  sample  k, 


B|*The  number  of  "successes"  In  sample  k, 


r^-The  probability  of  "successes"  In  the  kth  sample, 
and 


r0*The  overall  probability  of  "success." 

Also,  EAk  Ha  constant, 
k  * 

In  order  to  determine  which  cases  are  "more  extreme"  for  the  purposes  of 
calculation,  consider  that  the  Idealized  result  under  the  null  hypothesis 
constitutes  a  point  In  n-space,  where  n  Is  the  number  of  samples  being 
compared,  and  so  does  the  Idealized  result  under  the  alternative  hypothesis,  as 
well  as  the  actual  result  obtained.  The  square  of  the  geometric  distance 
between  the  result  obtained  and  that  Ideally  obtained  for  the  null  hypothesis 
Is  denoted  DA,  and  similarly  for  the  alternative  hypothesis  one  has  DB  In  the 
computar  program  which  was  developed  for  this  test.  The  test  can  be  adjusted 
for  unequal  sample  sizes,  however,  writing  a  general  program  for  this  Is 
difficult,  so  the  current  form  of  the  program  only  considers  equal  sample 
sizes. 

Of  all  the  ANOVA  methods  available,  this  method  can  be  the  most  Informative 
and  accurate  because  It  makes  no  distributional  assumption  (unlike  the  F-test); 
It  cannot  be  easily  fooled  by  random  effects  (unlike  the  Kruskal-Wallls  Test 
which  Is  basically  a  fixed  effects  test)',  and  It  lends  Itself  to  Inturpretable 
alternative  hypotheses  for  meaningful  power  analyses  (essentlelly  unlike  any 
other  ANOVA). 


I 


The  alternative  hypothesis  that  will  be  used  In  the  case  of  comparing  four 
samples  could,  for  example,  be  that  two  of  the  proportions  will  be  0.4:0. 6  and 
two  will  be  0.6:0. 4.  In  the  case  of  three  samples,  0.4:0. 6,  0.5:0. 5,  and 
0.6:0. 4  could  be  used.  This  Is  a  more  stringent  alternative  than  in  the  four 
sample  cases;  however,  the  odd  number  of  samples  makes  this  unavoidable,  and 
these  alternatives  are  relatively  easy  to  Interpret  and  communicate  to  the 
decision  maker.  For  future  reference,  In  this  example  of  alternative 
hypotheses,  one  has  W  =  0.40. 


APPLICATION 

It  was  found  in  reference  4  that  data  of  the  nature  to  be  gathered  for 
reference  2  may  be  decidedly  non-normally  distributed.  Other  distributional 
forms  could  be  experimented  with,  and/or  transformations  used.  However,  one 
will  not  be  certain  of  the  effects  of  such  manipulations.  In  reference  4,  the 
Kruskal-Wallls  One-Way  Analysis  of  Variance  (AN0VA)  was  used.  This  Is  a 
distribution  free  test  which  concentrates  on  location.  (It  Is  basically  an 
extension  of  the  Wllcoxon  Rank  Sum  Test  to  more  than  two  samples.)  From  this 
test,  In  an  example  given  on  KP,  no  significant  difference  was  found  between 
data  sources  as  far  as  location  was  concerned.  However,  It  was  obvious  that 
large  differences  In  dispersion  did  exist.  The  Kruskal-Wallls  Test  could  not 
discern  this,  nor  can  the  power  of  the  test  be  clearly  described.  For  these 
reasons,  a  k-stmple  extension  of  Westenberg's  Interquartile  Range  Test  and  the 
Westenberg-Mnod  Median  Test  (see  reference  1),  where  k  Is  greater  than  or  equal 
to  3,  has  been  derived  based  upon  work  performed  at  White  Sands  Missile  Range 
(see  reference  3).  Using  this  new  hypothesis  test,  both  random  and  fixed 
effects  can  be  Investigated  (dispersion  and  location),  and  the  probability  of  a 
type  II  error  (In  a  simplified  sense,  the  complement  of  power,  see  reference 
3),  will  be  provided  for  an  understandable  alternative  hypothesis.  (It  should 
be  emphasized  that  being  dl strlbutlon-free ,  bimodality,  etc.,  will  not  be  a 
problem.)  The  computerization  of  this  new  hypothesis  test  runs  Into  practical 
limitations  for  most  cases  when  k  Is  greater  than  5.  However,  the  Chi-square 
test  for  proportions  will  adequately  approximate  this  test  In  most  practical 
situations.  If  all  sample  sizes  are  20  or  better,  Chi-square  can  be  used  to 
compare  all  nine  samples,  or  even  all  18  when  considering  both  CONUS  and 
Europe. 


Evaluation  Is  proposed  as  follows:  the  LRXCAT  cells  will  be  compared  using 
this  new  test,  and  also  using  normal -theory  ANOVA  (which  can  be  Implemented  by 
use  of  the  SPSS).  If  the  results  are  substantially  Identical,  then  Duncan's 
multiple  range  test  can  be  used  (see  reference  7)  to  discern  which  LRXCAT's 
have  mean  values  which  are  Indistinguishable  at  this  level  of  testing. 

However,  If  the  results  of  the  new  vs  normal-theory  ANOVA1 s  are  not  compatible, 
further  use  of  the  new  test  Is  dictated. 

(Note  that  the  LRXCAT  cells  are  being  treated  Independently;  1 . e . ,  neither  LR 
nor  CAT  effect  Is  being  studied  separately.  Thus,  this  is  a  one-way  analysis 
of  variance.) 

Let  A  represent  LR1XCATI,  B  represent  LR1XCATII,  C  be  LR1XCATIII,  D  be 
'LR2XCATI,  etc.,  so  that  A  through  I  can  be  filled  Into  the  following  table  as 
shown: 
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CAT: 


IR: 

1 

2 

3 


I  II  III 

ABC 
0  £  F 
G  H  I 


(Not*  that  If  tom*  of  these  calls  do  not  exist,  this  analysis  will  not  suff*r.) 

Make  th*  following  four  s*ts  of  comparisons:  1)  A,  B,  0,  E;  2)  B,  C,  E,  F; 
3)  0,  E,  G,  H;  4)  E,  F,  H,  I.  If  a  «5%  Is  used,  for  «ach  comparison,  th«n 
the  actual  probability  of  finding  at  least  on*  "significant"  difference  Is 
between  5%  and  18.6%,  where  th*  latter  case  would  apply  If  all  of  th* 
comparisons  were  Independent.  (See  reference  5.)  Therefore,  In  reality,  5%  Is 
less  than  a  ,  which  Is  less  than  18.6%.  However,  given  that  there  U  on* 
comparison  (In  CONUS)  out  of  th*  four  enumerated  above  which  shows  a 
"significant"  difference,  purely  by  chance,  then  th*  probability  that  th*  same 

comparison  under  OCQNUS  will  do  likewise  Is  (  j  )(0.95)3(0.05)(0.25)  + 

(  ,  )(0.95)2(0.Q5)Z(0,5)  t  ( !  )(0.95)(0.05)3(0.75)  +  (  J )(0,05)4(1)  -  0.05. 

(This  assumes  that  CONUS  and  0C0NUS  are  equivalent.  Something  which  Is 
addressed  elsewhere  In  reference  2.)  Therefore,  th*  fact  that  comparisons  will 
be  done  In  OCONUS  on  the  same  LRXCAT  groupings  as  In  CONUS  can  be  used  to 
analytical  advantage  If  CONUS  and  OCONUS  are  Identical.  Also,  applying  th*  same 
reasoning  as  above,  If  Interactive  effects  are  Ignored  and  If  samples  can  be 
combined  first  by  LR,  and  later  by  CAT,  then  a  10%  significance  level  for  each 
comparison  translates  Into  a  10%  probability  that  should  on*  of  these  two 
comparisons  show  a  "significant"  difference  purely  by  chance,  then  the  same 
comparison  under  OCONUS  will  do  likewise.  Interactive  effects  can  be  handled 
as  on  pages  139-140  of  reference  1.  However,  the  more  comparisons  that  are  to 
be  made,  the  less  certainty  there  Is  In  the  stand-alone  analysis  of  either 
CONUS  or  European  generated  data. 

If,  under  this  plan,  B,  C,  E  and  F  appear  to  have  essentially  the  same 
variances,  but  none  of  th#  others  do,  then  their  variances,  (l.e.,  for  B,  C,  E 
and  F)  could  be  "averaged"  and  considered  as  equal  for  purposes  of  feeding  a 
model,  and  the  others  kept  as  distinctive  from  on#  another.  If  D,  E,  G  and  H 
appear  to  have  essentially  the  same  locations  and  so  do  B,  C,  E  and  F,  then  E 
will  be  averaged  with  th*  group  In  which  th#  comparison  was  most  significant. 
Further,  in  this  example  since  B,  C,  E  and  F  had  Indistinguishable  locations 
and  dispersions  (at  some  acceptable  level  of  power),  future  data  may  be 
collected  on  them  synonymously,  (If  It  Is  decided  that  the  r<sk  in  doing  this 
should  first  be  reduced  further,  then  Increase  th*  sample  sizes  In  the 
comparison,)  In  this  way,  th#  number  of  distinct  tables  of  organization  and 
equipment  (TOEs)  Influenced  by  distinct  sets  of  nonavallabl*  time  factor 
values,  can  be  determined. 

Note  that  although  the  extended  Westenberg  Interquartile  Range  Test  Is 
basically  a  random  effects  test  and  the  extended  Westenberg-Mood  Median  Test  1$ 


153 


. .  . .  ....  . ....  .  4  . 

. ■■  ia 


a  fixed  effects  test,  the  random  effects  test  can  be  confounded  although  this 
could  only  occur  In  cases  where  a  definite  difference  In  location  should  be 
found,  and  still  the  chance  of  confounding  the  random  effects  test  would  be 
small.  All  things  considered,  this  methodology  has  fewer  disadvantages  for 
this  application  than  any  alternative  available. 

The  fact  that  CONUS  and  OCONUS  units  will  both  be  studied  can  be  very 
helpful  here.  If  a  certain  set  of  LRXCAT  groupings  se#m  similar  in  the  CONUS 
study,  and  that  same  set  appears  similar  In  the  OCONUS  study,  then  this  would 
support  the  conclusion  that  they  be  labeled  that  wayi  if,  however,  one  set  of 
LRXCAT  groupings  appear  similar  In  CONUS  and  not  In  OCONUS,  for  example,  then 
either  the  CONUS/OCONUS  distinction  was  Important,,  or  the  Supposed'  si  rril lari  ties 
and  dissimilarities  may  have  been  by  chance.  (As  an  alternative  approach,  If 
two  sets  of  units  appear  Identical  In  CONUS,  one  may  pinpoint  those  two  In  the 
OCONUS  study  and, compare  them  specifically  with  a  two-sample  test.) 

The  following  will  also  be  considered:  If  two  observations  per  unit 
XCATXLR  grouping  can  be  taken,  where  “unit"  represents  a  unit  type  such  as 
transportation,  the  new  test  or  a  more  straight  forward  binomial  comparison  can 
yield  some  over-all  dispersion  and/or  location  Information  for  a  relatively 
large  number  of  unltXCATXLRs.  Th4  computer  programs  marked  WB1  and  WC  are 
designed  for  this  purpose  (see  Appendix  A).  Program  WC  can  be  used  to  analyze 
several  hundred  samples  of  two  observations  each. 
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APPENDIX  A 
COMPUTER  PROGRAMS 


A-l.  Program  marked  WA  1*  for  «  two-tailed  Fisher  and  for  three  and  four 
samples. 

A-2.  Program  WA1  Is  for  type  I  errors  only  (It  runs  faster  and  can  be  used  If 
the  "consumer  risk"  Is  unimportant). 

A-3.  Program  WB1  Is  for  type  I  errors  with  samples  of  size  2  when  there  are  up 
to  14  samples— computer  time  for  largir  number  of  samples  would  be  prbhlbltlve. 

i  *  *  ■  "•  *  1  i  .  i  .  i  .  , 

A-4,  Program  WC  Is  also  for  samples  of  size  2  but  can  be  used  for  up  to 
several  hundred  samples— note  that  the  hypotheses  are  different  from  the  other 
three  programs. 

In  these  programs,  NS  is  the  number  of  samples  being  considered,  IS  Is  the 
size  of  each  sample,  DA  Is  the  "distance"  of  the  observed  result  from  the  null 
hypothesis  and  DB  Is  Its  "distance"  from  the  alternative  hypothesis  where  100  x 
ViX  of  the  populations  from  which  half  of  the  samples  are  drawn  Is  found  outside 
(or  Inside)  of  the  Interquartile  range  of  the  combined  sample,  or  above  (or 
below)  the  median  of  the  combined  sample  and  vice  versa  for  the  other  half. 

For  example,  if  NS  *  4,  IS  ■  20,  and  W  *  0. 25",  then  the  null  hypothesis,  H0, 

and  the  alternative  hypothesis,  Mj,  can  be  represented  as 


10 

10 

10 

10 

10 

10 

10 

10 

5 

5 

15 

15 

15 

15 

5 

5 

If  the  observed  values  are 

7  8  12  13 
13  12  8  7 


then  DA  -  (10-7)*  +  (10-8)*  +  (10-12)*  +  (10-13)*  •  26  and  DB  ■ 

(5-7)*  +  (5-8)*  +  (15-12)*  ►  (15-13)*  ■  26.  Note  that  DB  Is 
calculated  using  the  order  that  makes  It  the  smallest  possible. 

The  programs  are  written  such  that  the  denominator  of  the  basic 
mathematical  expression  shown  near  the  beginning  of  this  paper  will  have 
rfl  ■  R.  If  R  Is  replaced  by  a  number  slightly  greater  than  0.5  then  xhe 

distribution  for  the  alternative  hypothesis  can  bw  "normalized"  so  that  tne 
area  under  the  representative  curve  Is  unity.  Some  values  for  R  are  given  In 
appendix  C. 

In  the  fourth  program,  WC,  the  number  of  zeroes,  NZ,  along  with  RA  and  RB 
are  explained  In  the  first  "FORMAT"  statement.  (Note  that  NZ  Is  the  same 
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number  as  DA  In  program  WB1.)  Tha  "FORMAT"  statement  Is  wrlttan  In  tarms  of 
tha  Interquartile  ranga  of  tha  comblnad  sample,  but  could  easily  ba  In  tarms  of 
tha  madlan  of  tha  comblnad  sample. 

PA  and  PB  have  common  meanings  across  these  programs.  PA  Is  tha 
probability  that  If  the  null  hypothesis  ware  true,  tha  result  obtained,  or  a 
lass  likely  one*  would  occur.  PB  Is  tha  probability  that  If  tha  alternative 
hypothesis  ware  true,  the  result  obtained,  or  a  lass  likely  one  under  these 
cbndltlons,  would  occur. 
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COMMON  F  (500) *0(300) *H( 300) *  DA* QB* I 1* 12* 13* 14# A*B* IS* W*R 
DOUBLE  PRECISION  A* Bi'DORP# ANS* AlS# AlC AP 

. WRITE  <6*  1100)  . ;  . ; 

1100  FORMAT ( 3X* 1  THE  FORMAT  IS  l 2* IX* 13, IX# F6.2* IX* P4.2* IX*F6. 
1  <1X*F«.6  l 

1  /5X*  *  FOR  NS*lS*DA*W*OB*Rt ) 

5000  RBAD(3*1*INQ«Z010)NS* I S*  DA* W*DB*R 

1  FORMAT ( 12* IX*  2  3* IX*  P6( 2* lX*  F4i 2* IX*  F6. 2* IX#  PB«6) 

m  -«»*.»l*»  l«*>^  'H  J  •«Al.»*l*«l  H«Hl  M . Ill-  -  *  1  ■  I  *■••••  ‘  l-.liK-  »-*>  *H»  n*i  m«  m  ****••  in  • 

L*I S*1 
DO  18  I  *  1*  L 
AA«I-1 

P <  I )  ■  (0i54A-AA)*(0'.5#Ai-AA) 

0(1)  ■  (W*A-AM'*<W*AiAA) 

. . intm  ••*“•(  (  V.O*W)*A«AA lin'd  1»0-W)*A-AA)‘ . . . . . . . . 

A»0 

uro 

c  «0 

ISfNS/2 

CO  TO  <1000*2# 3, 4)NS 

nooo'  •WRITE- (6>  l00iV  - . . . . . . . .  . . . - 

1001  FORMAT  <  SX*  *  BAO  VALUE  OP  NS') 
on  TO  9000 
■1  UO  30  11*0*  IS 
on  30  12r*0*  IS 
t3  ■  J  -  II  -  12 

. “'lF(U3.LT.0r;0«;<l3,(5T’.IS>)  "00  TO  31  . * . 

CALL  SUB3 
31  CONTINUE 
30  CONTINUE 
70  !3[]KP«1.0 

1C *\F  T  ■  <-N$*IS/2>-l 
■  ■  DO  71  ICAF , •  O.ICAFI 

ANS.NS 
A I S  ■  I S 

a1CAP«ICAF  . 

7)  DORP  *  OURP  4  ( ( ( ANS4AIS/2.0) -AlCAF I / ( ( ANS4A I S)«AlCAP ) > 
PA  •  A4D0RP 

— . -PB  »  8«00RP  . . . . 

GO  TO  2000 
4  00  130  1  1*0*  I S 
Ofl  110  12-0*  I S 
DO  130  13*0*  IS 
14  •  V  -  II  -  12  -  13 

.  I F  <  ( 14  ,  LT«  0) '.OR.  (14 ,6’T'.  IS  ) )  GO  TO  Hi  1  . 


3 

44 . 

CALL  SUB4 

3 

47. 

131 

CONTINUE 

3 

48. 

130 

CONTINUE 

49. 

CO  rn  70 

50. 

2 

on  230  11  ■  0*!S 

t - 

. “31,  ' 

. — . 

12  •  .1  -  11 

1 

5P. 

IF  ( ( 1 2 .LT.O  )’.0R,  ( 12 ,l 

1 

51. 

CALL  SU82 

1 

34. 

231 

CONTINUE 

1 

35, 

230 

CONTINUE 

34. 

CO  TO  70 

..  ■  57t 

.  2000 

UR I  TF.  ( £>* 2001 )  »A*PB 

5P , 

2001 

FORMAT (iX*F9,4*3X*F9 

39. 

GO  TO  3000 
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60 

61 


2010  STOP 
END 


62. 

63. 

66. 

69. 

66. 

67. 

'68, 

69. 

70. 

71. 

72. 

73. 

74. 

75. 

76. 

77. 

78. 

79. 

80. 
61. 
82. 
83. 
86, 

85. 

86. 

87. 

88. 

89. 

90. 

91. 

92. 

93. 

94. 

95. 

96. 

97. 

98. 

99. 
100. 
101. 
102. 


SUBROUTINE  SUB 3  '  — " 

COMMON  F<500)/0(500)jH“<500)iDAjDB#  1 1  j  12*  II*  14#  A* B#  IS*  W«R 
DOUBLE  PRECISION  A*BJPROO#X,Y 
m  -  n+i 
1 1 2  *  1261 
113  -13+1 


DAA  ■  P( II1)+F(!!2)+P( lit) 

DBBl  ■  0(im*P<Il2)6H(IIl> 

DBB2  ■  G(Iin6P(Il3)*M'(ll2) 

DBB3  a  OtII2)6P(lIl)*H’(lil> 

0BB4  a  G( 1 12 )*F(  1 13  )*H'(  III) 

DBB5  a  G(I!|)*P<!!l)*H'(lit) 

UBB6  ■  Ot II3)*F( Il2)*H‘(  III) 

OBB  a  MIN(D3B1«'DBB2jBBB3jDBB4jDBB3;DBB6) 
PROD  a  1,0 


IF'(  1 1  »EQ.  IS)  GO  TO  92 

in  ^  is  -l  -n 

on  31  IFAC  •  Oiin 
X  ■  IS. IFAC 
Y  a  X  «  II 


31  PROD  a  PR006X/Y 

32  1FH2.EQ.IS)  GO  TO  34 

1112  a  1S-1-12 

DO  33  IFAC  a  0«  Z  X 12 
X  ■  IS-IFAC 

Y  a  X  .  12 

33  PROD  a  PROQAX/V 

34  IF  ( 23, EO, IS)  GO  TO  36 

1 1 1 3  •  IS-1-I3 

DO  53  IFAC  ■  Oi 1 1 2 3 
X  a  IS. IFAC 

Y  »  X-I3 

33  PROD  ■  PRQ06X/Y 
36  2FtDAA.LT.DA)  GO  TO  61 
AaA+PROD 


61  IF(DBB.GE.DB)  GQ  TO  62 

RETURN  •  •  . . 

62  B  a  B+PRUD*{ ( ( ( ( 1,0/R)*( 1.0-W) ))•*(! 1*2) )*((<{ l,0/R)*Wj ) 

1  **(13*2)1) 


RETURN 

END 


103. 

104. 


SUBROUTINE  SUB4 

COMMON  F(300)/G{300);H(300),DA,DB,I1;I2#I1#I4#A#B#1S;WjR 
158 


J  4l  jiiV.t:  t  -  — '  T  -/  I  Lj 


DOUBLE  PRECISION  A,BiPROO/X>Y 
111-11*1 
m«i2*i 
1 1  J“!3*l 
II4»!4*l 

DAA  •  F(III)+F(II2)+F(II3)+P(I14) 


DBB1  ■  0(  1 1 1  >  *0(  1 12  )*H’{  Ill)*H(  1 14) 

DBB2  »  0(111) *0 { l  II)  *H'(  l  !2)*H(  1 14 ) 

DBBJ  ■  6(lU)*0tU4)4H'<I!lUHlIH) 

0BB4  ■  G(Il2)*0niS)*H‘<Iti)*Mm4> 

"*onr  0(ii2i*o{ii4)*H*(itn4H<m> 

■  DBB6  ■  OU  I J ) *G ( 1 14) *H‘{  1 1 1 ) *HU  12 ) 

DBB  ■  MIN(0BB1jDBB2,DBB3jDBB4jDBB3/DBB6> 

PROD  •  1.0 

IPJIi.EQ.IS)  00  TO  192 
II!  ■  XS-l-X 1 

“‘DO"’ I'll" IPAC'  AOilll  .... 

XilS-lFAC 
Y*X-Il 

,  PROD  ■  PROD  *X  /Y 
!  IPII2.EQ.IS)  00  TO  194 

1112  •  I S"l-1 2 

-w-m-me  •0*1:12  . 

X  «  ISmIPAC 

Y  *  X«I2 

I  PROD  ■  PROO  *X  /Y 
►  IPII3.EQ.1S)  CO  TO  196 

1113  ■  IS-1-I3 

■*W~199  IPAC  *0* 1 113  . 

X  *  IS-IFAC 

Y  «  X*13 

l  PROD  ■  PROD  *X  /Y 
i  IP  (I4'.EQ,IS>  CO  TO  199 
IIU  ■  I S- 1-14 
-DO’  13?  IPAC  *0i  1114 
X  «  IS-IPAC 

Y  ■  X-I4 

'  PROD  ■  PROD  *X  /Y 
I  IPIDAA.LT, DA)  00  TO  161 
A«A*PROD 

rl'PtDBBVOe.DBl  60  TO  162 
RETURN 

!  B  *  B*PROD*U(il.0/R)*(l.O-W))*#nii*12)*2))*lm,O/R)4W>4* 
l  ( (13+14)42)  ) ) 

RETURN 

END 


SUBROUTINE  SUB2 

COMMON  P(900>/Of900);H(900)#DA*DB/!i;X2*!l»!4»A#B»!S#M«ft 
DOUBLE  PRECISION  A.B.PROD/X.Y 
lli*li+l 
1 12*12*1 

OAA  ■  F(II1)  *  P ( 1 1 2 ) 

OB Q 1  -  G( 1 1 1 )  *  H ( 1 1 2 ) 
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raiimu  ■ r 


3FTN#S  K,WAl/tKEK 

FTN  1  OK  1  01/09/S2-lll57<6i#) 


AO. 
“ttv 
42. 
— Atr 
44. 


COMMON  F{200)#DA#I1#!2#13#14#A*IS 
DOUBLE  PRECISION  A. DORP# ANS# A IS#  A I CAP 

- wtmrr-rvmoot - 

1100  FORMAT <  5X# • THE  FORMAT  !S  I  2#  IX# 13, U# F6 . 2 

. vm/i  FOR  NS#1S#DA>> 

9000  R!AD(5#l#ENDa2010)NS#!S#DA 

- t~ftmtttrmTixiTJAixiF6.n . .  — 

A*  IS 

- cmn - 

.00  16  I«l#l 

— : — aw«t - - - ; . -  . . “ 

_  16  FU l  a  (0,3»AxAA>*(0.9*A»AA)  _ 

J«IS*NS/.J 

- -oo  ?ff . r  yrogrtOTpnyitWs - - - 

1000  WRITE  Ujiiool) 

”n^Ol^t3RMA1^'9Xm-A'0-VAtOE“13F-1«Tl - 

00  TO  5000 

- *-W‘"iO“tt«om - 

DO  30  I2*0«  I S 

- 13  a-.j-  w-fT  n-ta - - - 

IFUU.lT.Df.OR.tU.OT’.ISn  00  TO  II 

. vAtt'  "SUBJ" . .  . . . - 

31  CONTINUE 

. "'30  ‘CONTINUE . . . . . . : - - -~ 

70  DORPal‘,0 


00  71  XCAF  a  0#lfcAFI 

- ANS'aN* - - - - — . . . .  * 

A3S-IS 

- A1  frA  F  a  I CAF - — . . . .  '  ~  '  ' 

71  OQRF  a  DORP  A  ( [  <AN5*Ai  S/2.0I-AIC  AF )  /  ( ( AN$*A1  5 ) -AlCAF )) 

- IA  a  AMpOrR'F - 

GO  TO  2000 

— t4~o  o  — rstj — rir»T>Tit'S - 

DO  130  X  2-0#  X  5 

- DQ~~lSO-i3-*0>lS  ' — * - 

14  >  J  -  II  -  12  -  II 

- 1 F  It  T4".  L T 70 )  *.  OR 77 t'«TOT ‘.IS})'  00  TO'  1 1 1 - 

C  A  CL  SUB4 

iH" CONTINUE""  . . .  . . . — 

130  CONTINUE 

- -GO ‘TO-  TO'  - .  ‘  - .  . 

2000  WRITE (6/ 2001 )  PA 

2001  ■PORHATt'l'XJ'FO  .47 - 

GO  TO  5000 

■2010’ STOP  . . . - . . .  -  . . 

END 


COMMON  F( 200 1; DA# 11# 12; 13/ 14; A; IS 

double  precision  a#proo#x#y - 


<-i -V'-’U. iVJiCi«'^i'i-.-.\u,_ivi' ii  ,>,i.  ^ 


I»>iw  HWIMIPI V»M3 WWSPftBBIWPW 


111  •  11  +  1 

1 1 2  ■  12  +  1 
113  »I3+1 

DAA  ■  F  ( 1 1 1 )  +P it  1 12)  +P ( 1 13) 
IF< DAA.GE ■DA)  GO  TO  61 
RETURN 

61  PROD  ■  1*.0 

IF ( 1 1 ■ EQ» IS)  GO  TO  92 

in  ■  is  «i  .u 
GO  si  IFAC  ■  Oi ! ! I 
X  •  1S-IFAC 
v  •  X  -  a 

51  PROO  ■  PR0D6X/V 

52  IF<I2,EQ'.1S>  GO  TO  SA 

1 1 1 2  *  IS-1-12 

DO  S3  IFAC  •  Oj  I X 12 
X  A  IS-IFAC 

Y  »  X  «  12 

53  PROD  ■  PROO*X/,V 

SA  IP  <13, EG, IS)  GO  TO  56 

1113  >  IS-1-I3 

DO  S3  IFAC  ■  Oj  X 1 13 
X  ■  ! 5-1  FAC 

Y  •  X-J3 

33  PRUO  ■  PR006X/Y 
3b  AaAAPROO 
RETURN 
END 


SUBROUTINE  SUB4 

£5f!2?»  F<200>*DAiIl/!2;iS/X4iA,  I 
DOUBLE  PRECISION  AjpROD/X/Y 

111- U+1 

1 12- 12*1 

1 13-13*1 
IIA»14+1 

DAA  ■  P(IXX)+P< XI2)*P( jlS)AP(  IIA 
IF(DAA.OE,DA)  GO  TO  16) 

RETURN 

161  PROD  ■  1.0 


IHil.EQ.IS)  GO  TO  132 
III  ■  IS-1-I1 
DO  131  IFAC  -OiIXI 
X«IS-IFAC 
V-X-X 1 

131  PRUO  ■  PROD  *X  /Y 
152  IFt 12,EQ> IS)  GO  TO  ISA 

1 1 1 2  ■  IS-1-12 

DU  133  IFAC  «0j  X 1 12 
X  ■  IS-IFAC 
Y  ■  X-I2 

133  PROD  ■  PRUO  *X  /Y 
ISA  IF< 13, EO. IS)  GO  TO  136 

1 1 1 3  ■  I S  —  1— X  3 


162 


'WHWI.HIt,  (1WPMWIMMUPB-W . . 


1 

1 

l 


101. 

106. 

107. 

106. 

"TOT." 

110, 

-mi¬ 

nt. 

“titr 

m. 


on  155  IF  AC  >0<|lt3 
X  *  X S-IFAC 

y  ■  x-n 

155  PROD  «  PROP  »X  /Y 

19  1116  •‘iS-lJIi^0  TDl5* 

- D0~157‘"1FAC  .Oil  114 

X  F  IS-1FAC 

- - - - . 

157  mao  ■  prod  *x  /y 


1 

* 

i 


I 

i 


il 

C 

l 


Ip 

ft 


[if- 

t 


l*: 


I 


i 


SFTN*S  K.WBWKEK 

PTN  10(<1  01/05/82-11157(43.) 

1 ,  COMMON  F  <200)/ DA* 1 1, 1 2, 13, 14*13* 16, 17/ 18*' 19*  IlO*  U it  1 12*  I 13/ 

2.  1  114# A*  IS 

- - UOUK-F  PRTC I  5TUN“A'*'D0RP'*ANS*  A2 S/ATCAF - 

4.  C  READ  AS  MANY  CARDS  AS  DESIRED*  ONE  AT  A  TIME 
9,  C  END  BY  USING*!* 

6.  WRITE  (6*1100) 

'  * . . T.1  1100“  FORMAT  1 5X#r’TH8  "FORMAT  !*’ . 12#IK*P6.2  » - - - 

8.  1  /9X*  •  FOR  NS, DA  *) 

- n - 5'00T3‘  RfAT)75717TND»Z,fflO)lTSTD’I - - - 

10.  1  FORMAT (l2*lX*F6t2*iX*F4.2*lX*Fb.2) 

. -  - — n*;“ . .  - 1  r* '  2 . . . .  . . . - . 

12.  A. IS 


14.  DO  1 18  1-1#  L 

“I - m - WIFI - 

1  16.  11B  F(I>  «  (0i**A-AA)*<0’.5*A»AA) 

- rT_ - *rO - - - * - 

18.  J*I$*NS/2 

- tt- - oo"TO~n'OOo»lotroyiooo*irn'ODDT6yi'OOo#Tnt)oo>io>ipgo*ii,*,‘' 

20.  1  1000# 14  INS 

- nr - row-yRTrt"'T67tool') - - - 

22.  1001  FORMAT  (3X**0AD  VALUE  OF  NSM 

. — '22'.'" - - ~00~TD“ 30150“ - - - - -  - - - - - - - - - 

24.  70  DORPAl’.O 

- 2jf~ -  - ICAR  I  ftlSFlS’/T)'*! - - - - - - - 

26.  DO  71  I C AF  •  0* 1CAFI 

-t - m - - — - 

1  28.  A 2  S - 2  S 

*1 - Jfr-  .  A1CAF-ICAF .  . . 

1  30.  71  DORP  »  DORP  *  ( ( ( ANS*AiS/2.0) -AICAF )  / <  ( ANSmAlS l-AICAF  > ) 

. . Jli . PA  ■  A*DORP  . 


32. 

- it; — 

1  34. 

■■  1 - 15  r — 

3  36. 

3 - •37.  -- 

3  33. 

- , - 137 - 

3  40. 


2 

~T 

4 
-5 

5 

9 


V 

2 

3 

~*r 


42. 
—417- 
44. 
—437“ 
4b. 
— 47. 

48. 

“44. 

90. 
—31V 
52. 
— 53. 

54. 

“35. 

5b. 
~  “97. 
58. 
34. 


GO  TO  2000 

—»”Dt)-no'n»o*is - 

DO  130  22-0# IS 

- Do~no-n*orrs - : 

u  ■  j  -  n  *  iz  -  i» 

-  ‘IF1  ( !4.iT'.0)f.0R(1 14.GT'*1S) )  GO  TO  111 
CALL  SUB4 

llftONTTNUg - 

130  CONTINUE 

- 00"3fl-70 . . . — 

b  DO  330  2  1 -0#  IS 
DO '330  12*0# IS 
DO  330  13-0# IS 

- pp-»o  TMOiTS - 

DO  330  IS- 0# IS 

lb  •  J  -  II  •  12  -  IS  i  14  -  15 
IF  t  Ub.LTtO)  .OR.  ( lb. GT. IS) )  GO  TO  »!l 
CALL  SUB6- 
331  CONTINUE 

tio-cotmtujf . . . . 

GO  TO  70 

—8  00-330-1 1*0*  IS  . 

DO  530  12-0# IS 
DO  530  13-0#  IS 
00  330  14*0*  IS 

- DO  “330  I3»0*TS - 

DO  530  16*0*  IS 
DO  590  17*0# IS 


164 


5 

-6 


7 

60, 

18  •  J- 1 1-12 

T 

61. 

IF(<I8.LT,0) 

■  7 . 

62. 

CALL  SUBS 

7 

63. 

531  CONTINUE 

7 

- w... 

i Ig  CONTINUE 

65. 

GO  TO  70 

GO  TO  5)1 


1 

~r 

) 

T 

S 

V 

7 

-f 

7 

f 


56  , " 
67. 

Ktr 

69. 

"TOT 

71. 

ttt 

7*. 

-76T 

75. 

"TOT 

77. 

Ttr 

79. 

-tor 

si. 

~tfr 


10  00  750  11*0/ IS 
DO  750  12*0/1$ 

— do  noitwons . 

DO  750  19  •  0# X8 


- DO"  750  TW)H" 

00  750  16*0#  IS 

- 00-710  17*0/18 - - — . . . - 

DO  750  18*0/  1$ 

- D0“750-T9*0iiT5‘- . . . .  ‘  '  -  -  ' - 

110  •  d-ll«I2«!l«I4.!5il6-l7-IB-19 

mrmTtTror .or. ctiotgtl.  mr  GtrnrtTt 


. .  i.wt  ro7. 
CALI.  5UB10 

Ttr-tONTINOt - 

730  CONTINUE 

- 00™T0'~70' 

iz  do  tso  ii-o#a 

irwc-mw 


9 

t- 


2 

5" 

6 

— 5- 

6 

B 

-7- 
10 
-ti¬ 
ll 
If 
11 
Tt - 


B3. 

■Tlr- 

B5. 

-86/— 

•7. 

16. 

S9. 

-90“.— 

91. 

"~92", 

93, 

-99-.— 

95. 

“967 


DO  830  13*0/2 
'‘OO  “t30"l'A“  l»— 0I2‘ 

DO  830  15*0/2 
DO  830-ttf«0>-r- 
00  830  17*0/  Z 

'-BO-tyr  ia»ojz - 

DO  830  19*0/2 

“U0'  830  ‘110*0  J'2 - - - . . . 

DO  830  Ill*0i2 

lit  *-j-iwj  i»:3-i*-i3-n-i7-ia-i9.i  io.it  i 
IF  C (I12.LT.0). OR . ( 1 1 2 • GTh  2  > )  GO  TO  831 
-oitrt — susi2  — - - - 


1 

T 

3 

-Tr¬ 


ail  CONTINUE 
“830“C0NTttmr- 
97.  GO  TO  70 

-90s - IT -00-930  -  IT«0>  2" 

00  930  I  2*0#  2 
"00-930 -1 3  ■Oi  2 


99. 

*1007 - 

101. 

-lot-,- — 

103, 

tOTj - 

105. 

■106; - 

107, 

Ir- 
109. 

-no.-  - 

111. 

tTf, - 

113. 

-119; - 

115. 

“116—— 

117. 

irr; — 


7 


DO  930  19  •  0/2 

— 00-930“  It* 0*2 . 

DO  930  16*0/2 
-“00-930  1 7*0/ 2 
DO  930  18*0/2 
— 00-930  19*0/2 


11 

-tt- 

13 

16- 

13 

lt- 

13 


DO  930  1 10*0/2 

00-930- ti-l-O/Z - - -  - -  ••  - 

DO  930  X12*o;2  ‘ 

00-930-115*0/2 -  - - - . . . - . . 

119  ■  1 1*1 2* 1 3* 14* IS* 1 6* 1 7* 1 8* 19*1 £0*1 £1*1 12* J 13 
iF<tix».LT.oi«Bn.  rmrcTVttt -ra  Tout - 


CALL  SUB19 

— 93lr-CONTiNU!! - 

930  CONTINUE 

- -00-70  70 . . . - 

2000  WRIT! (6/2001)  PA 
■tOOl  'FORMAT!  txrft. 9  >  “ 


119.  GO  TO  3000 

-WO-.—-  2010  STOP 
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122.  SUBROUTINE  SUM 

- ur - common  'TTzotfttwni;  nrnntnB7ii«  i*/h;wti6i  HI#  fiinrn 

12*.  1  I 1*>  A* IS 

125.  oojbCe  PRECISION  a*x;y7prdd  ' . 

126.  lli*!l*l 

117.  111*12*1  . . . 

126.  1 1 3*  t  3*1 

- 177: - rmrtf*i - : - 

130.  OAA  ■  F(IIl)*Pni2UP(il»)*P(II*) 

131.  IFIDAA'.GE.DA)  OO  to  106  . . . • 

1 3  2 .  RETURN 

113.  100  PROO  ■  l'.O  - . . . 

13*.  IPdi.EQ'.IS)  GO  TO  152 

— - w.~~ . t  rt“»-TS«l»Tf - - - 

136.  00  131  IPAC  ■0*111 

l  137.  X-IS-IFAC  . . . 

1  138.  Y-X-X 1 

1  139,  151  PROD  ■  PROO  *X  /V  . . 

1*0.  152  IF ( 12  iEQ.  IS )  GO  TO  IS* 

. . t*r. - - ni2  *-n-i*i2  •  --  * . - - - ; - 

1*2.  00  133  IPAC  «0*‘tII2 

l  1*3.  X  ■  IS-IFAC 

l  1**.  Y  ■  X-I2 

1  1*3,  153  PROO  ■  PROD  *X  /Y 

1*6.  15*  IF(II.EQ.IS)  GO  TO  156 

- -  1*7.  . 1113  •  IS-1-I3  . . . . 

1*8.  DO  155  I  FAC  •0*1113 

l  1*9.  X  *  IS-IFAC  - 

1  150.  Y  ■  X-13 

1  151.  153  PROD  ■  PROO  *X  ,Y 

152.  156  IF  ( I  * . E  Q • I S )  GO  TO  158 

. . ‘133; - - Ill*  ■  IS-l-I*  -  * - - 

13*.  00  137  I  FAC  *0*111* 

l  133.  X  ■  IS-IFAC  . . . . 

1  136.  Y  •  v-1* 

l  137.  137  PROQ  ■  PROO  *X  /Y  —  ~ 

158,  138  A*A+PR00 

- 133.  ■  RETURN  . . 

160.  END 


161. 

162. 

163. 

16*. 

163. 


SUBROUTINE  SUB6 

COMMON  F ( 200) *  DA* 1 1* 1 2* 1 3/ I** IS* 16* It* 18* 19* ! 10* 1 1 1* 1 12*  1 13* 

1  1 1** A* IS  - - - 

DOUBLE  PRECISION  A*X*YiPR0O 
111*11*1 


166 


170- 

171, 

iTtr 

ITT* 

179. 

Tftr 

177, 

-rrtr 

179. 

"tftOr 

m;. 

-mf 


-*00' 


DA4  •  PUU)  ♦  P<Iia>*P'<IIl)‘>PUI4)+F,Ul5)+PUl6> 
tft*A*Y&t  iOAl  00  TO  100 
RITUR5*  . 

PROO'imVo — . . . 

IPUl.EQ*.  IS)  00  TO  192 


rrrrrsjun 
do  m  ipac 

'X-TT-tS-TPAC  • 


«  o,m 


lit 

ist 


TT 
V  •  X 

■PROD 


lit* 

rtttr 

199. 

•Wr 

117. 


.  ~PRbo-*rvr 

IP  1 12. IQ*.  IS)  SO  TO 

nti  nwrn - 


294 


00 


TT 

199  IPAC 
ii-lFAtr 


•  0*1112 


■tttl 

199. 

190.- 

191'. 

-in?- 


V  ■  X-12 
-♦It-PROD  y-pROO -*K  /V 
194  IP  ( tl.EQ. IS )  00  TO 


194 


tor, 

195. 

t96v 

197. 


WfrWwtf 

00  195  IFAC-0#iltl 

. -X-r-lSilPAC' 

V  •  X  •  11 

15t-  PROD  -•  PROD  *X  /V 
15*  IP‘{  14. EQ.  It)  GO  TO  159 


lltf  Hll«l«l» 

DO  197  IPAC-0*  t  X 1 4 
p-r^rrtPAC  . 


t99~. 

199. 

-toof 


Y  •  X-I4 
15T  PROD  ■  PROO  *X  / Y 
159  IPdI.eQ'.IS)  GO  TO  1*0 


-toot- - mo  mnny 

201.  00  259  IPAC-0#tlI5 

— . XrrtSntftZ . 

Y  hX-15 

-159  PROD  ■  PROD  *X  /Y 
205.  260  IPU4.9Q.IS)  Go  TO  262 


tor. 

201. 

t09r 


t06.~ 

207. 

109“ 

209. 

rtwT 

211. 

•Mr 

211. 


it  to  -rs-t-wir 

00  141  IFAC-0*!I16 

. X-r  -  iSktPAe 

Y  ■  X-16 

-m-PROO  •  PROD  *X  ('Y 
162  A-A*PROO 
- MtMM - 


END 


- .  SUBROUTINE  SU19 

215.  DOUBLE  PRECISION  A.X.'Y.PROO 

- - OOMMONPfEOOWOA*  limit!*  14#  15*  !**  17*19*19*  1 10*111#  112*  113* 

217.  1  1 14* A. IS 

219.  111-11*1 
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112-12*1 

113- 13*1 

114- 14*1  r— 

115- 13*1 

.  m«r6*i . . . . . . . ; - 

117-17*1 

na-i8*i  '  - -  - - 

DAA  a  F(IIl)*P(II2)*F(ilS)*P(II4>*P(tI5>*P(II6)*PUI7)*PllI8) 

1F'<DAA.GE,DA)  60  TO  506  - - - 

RETURN 

5  CTO  "PROD  ■  170 . . . . 

iF(II.EQtlS)  00  TO  582 

m-is-i-n  . . 

00  561  I  FAC  *0*  III 

X  a  tS-IFAC  . . . . 

Y  a  X-ll 

581  fROD  T  PROD  *X"/Y" - - - - 

562  IFU2.EQ.IS)  GO  TO  564 

Z 1 12  -  IS-1-X2  . . 

DO  563  I  FAC  iO;X!l2 

X  a  IS-IFAC  . - . . 

Y  a  X-X2 

363"  PROD'  *'  PROD'  *X“/V - - - 

564  IFUB.EQ.15)  GO  TO  566 

1 1 1 3  «  18*1-13  . . . 

DO  565  IFAC-O* 1113 

X  a  1S.1PAC  .  . 

Y  a  X-1 3 

565  PROD  r  PRDD6X/Y . . . 

366  IFf  !4.EQ‘.:S)  GO  TO  568 

1114  -  18-1*14 

DO  367  2  FACaO* 1 1 14 

X  -  IS-IFAC  . . 

Y  a  X- 1 4 

367  PROD  *  PRQO  *X/Y  - - 

568  IF (  IS.EQ'.IS)  GO  TO  570 

1113  ■  18*1*15  - - 

DO  569  I  FAC  -  0« I Z 15 

X  a  IS-IFAC  . -  - 

Y  a  X-I5 

369  PROD  •  PROD  *X/Y  . 7 - - - 

370  ZF(Z6aEQ.ZS)  GO  TO  372 

1116  ■  IS-1-I6  . - - - — 

DO  571  IFAC  *  0*1116 

X- IS-IFAC  . . . — . . 

Y.X-16 

371  PROD  ■  PR0D6X/Y  . . T - 

572  IFU7.EQ.1S)  GO  TO  574 

1 1 17  •  IS-1-I7  . 

DO  573  I FAC-O* 1117 

X-1S-IFAC 

Y-X-I7 

373  PROD-PROD*X/Y  - - - - 

574  IFU8.EQ.XS)  GO  TO  376 

1118  ■  IS-1-I8 

DO  575  I FAC-O* 1 1 1 S 

X- I S*1 FAC 

V-X-IS 

375  PROD  ■  PROD  *X  /Y  - 

576  A  a  A+PROD 


’’I 


— ztxr. -  END 


281,  SUBROUTINE  SUHO 

m? - eowNON-ytsoo  i  i  rai  ttyirc;  m tw ns  nsits mtr/i  rarnry tit*  tn> 

211.  1  H4jA*I8 

- - ftti - OOBBtf  PRECISION  AjXJyJIPROO 

219.  Itl'll*! 

— ; - ttti - m«mr .  ..  .. 

2BT,  xxs»ts*i 

- ^ - ntwWr - 

IBS.  1 1I«I9*1 

- - 290i - . - . . 

291.  I!T«I7*l 

- ttt. - ttt6ft*i~ .  1 

29S,  I I9«|9*l 

- - ni  onion  - - - — - - - - 

299.  ORA  »  P<ltt)*P<112UP<IIS)*p(m>+F(fl9>*P(n6)*FUX7)*FllI8) 

—  - m-; - 1— . *mi9>*R<moS 

297,.  1FI DAA.OE.OA)  00  TO  71! 

- 29tr . . . RETURN . 

299.  7B1  PROS  ■  I'.O 

- twi - f-mf-yBOVisroo  ■  To-m - - - - 

sol.  in  »  t s.i.i I 

- mi - DO  761  IPAC . *0.1  II 

l  SOI.  X  »  1S-IPAC 

— 1 - SO*-. - - -Y-r'-X-ll . 

1  S09.  761  PROD  •  PRQ06X/Y 

- son - Ttriftir.n'.li)  •crwwv . - . . — . 

SOT.  11X2  *  IS-l-12 

- tot'. . . -OO-Ttr-IRRC  fOilltZ 

1  109.  X  *  1S-IFAC 

— t - - - - Y  *  *-12 . 

i  111,  76*  PROO  ■  PROOAX/V 

- mr, - Ttr-tftrmtittt  oo-  to  T»er - - ~ - — ----- 

SIS.  Ills  •  15*1*13 

- Ittf: - .00-769-  1PAC  »0>  1 1 1 3 

1  S19,  X  *  tS-IPAC 

» - tt6i - Y~w  X*TS 

1  117.  761  PROO  ■  PR0D6X/V 

- SitT - 766 -'tP  (  1A.BQ.  IS)  00-T0--768  .  . . . . 

S19.  HU  •  15*1*14 

- m>  . . 00  767  IP  AC  »o;  I II 4 

l  S2 1 •  X  ■  tS-IPAC 

— 1 . . tttv . . . V  »  X-16 

1  SIS.  767  PROO  •  PR0D6X/Y 

- mi - rot  -tmtitortti-oo  to  770 . 

325 .  1  1  29  *  I S* 1*15 

- “ - -ttti - 00-  769  I P  AC«0#  1 1 1 9 

l  S27.  X  •  IS-IPAC 

-  i .  -  -li  t. — . -  Y  *  X-I9 

1  S29’«  766  PROO  •  PROOPX/Y 

- - TfO~lft  r6rt0ittr  00  TO  772 

SSI.  1116  ■  15-1*16 

. .  SStv  . DO  771  I FACaO*  1116 

169 


x  -  IS-IFAC 

Y  ■  X-I6 

771  PROD  ■  PR004X/V 

77?  I F ( ! 7 * E Q • l S )  00  TO  774 

. . TTirnispn - - 

00  773  IPACvO# i 1 17 
X  •  IS  •  X P AC 

Y  «  X-I7 

•771  PROD  4  PRODiX/V . . 


mmmllusii 

Ittfritaiafftti  TMinHi 

144. 

00  775  IPACiOsillS 

345. 

X  1  IS* IP AC . 

346. 

y  ■  x-ie 

. 347. 

775  PROD  ■  PRQD4X/Y 

348. 

776  IPin.EQ'.IS)  60  TO  778 

I'A'P", . 

-'TT  T9r~r-T3'#TiT9 - - 

350. 

00  777  IF  AC  «  OsIIH 

. -151." 

. X-#-  IS- IF  AC  . . 

352. 

Y  .  X-I9 

. 153. 

777  PROD"*  PR0D4X/V  . . 

354. 

778  X F ( 1 10. EOi IS  1  00  TO  780 

1 1 ! 10  *  TSrrslTO 

356. 

00  779  iPAe*o#iiuo 

357. 

X  «.  tS-XPAC 

358. 

Y  F  X-UO 

359. 

779  PROD  •  PR0D4X/Y 

360. 

- 3  61'; . - 

780  A  s  A  *  PROO 
. . "RETURN- — . - . . . . 

SUBROUTINE  SU812  . . 

COMMON  F  ( 200  Is  DAs  I  Is  12#  13#  14#  1 5s  16#  Its  18#  1 9#  I 10# I X I; ! it# ill# 

1“ . IV4#A#TS - - - 

DOUBLE  PRECISION  A#X# YsPRQD 

ni*u*i  . . 

1 11*12*1 

1 13-13*1  - - 

114*14*1 

It*  **1-5*1 .  - . . 

114*16*1 

117- 17*1  .  -  - . . 

118- 18*1 

119-19*1  . . . - 

iiio*iiu*i 

immui  - - 

1112-112*1 

oaa  *  F<im*FU!2>*P!m)*p<i!4)*p<!tjupu!8>*ptim*Tlrr»r 
1  *F(IX9J*P{II10»*P(Ull)+P(IU2l 
IF(DAA.GE.DA)  00  TO  t6l| 

RETURN 

I  PROD  ■  1.0  . . 

IPII1.EQ.2)  00  TO  2542 
ill  *  i-ti 


417. 


00  2961  I  FAC  *0« HI 
X  ■  2-IFAC 
. V  •  X-Il 

2961  PROD  •  PRQDWX/Y 

2962  IHII.bQVrr'OO  TO-  2  964 
1112  •  1-12 

-  .  00  2161  IP  AC  *0>  1 1 1 2 

X  i  l-IFAC 

- y-r-*wj2 . - . - 

2961  PROD  •  PR0D6X/V 

2964  I  Ft  1 1 V50V2 1  ~CIT Ta~2I6‘6 . 

1113  •  1-13 

-  . DtriJ6»  TPAC  wOillll 

X  ■  2-IFAC 

- -y-i-Jt-rs . . 

2961  PROD  •  PR006X/V 
2966'"  tft'f tQ '.'2  t-CO-TO’  "2 966 

1114  •  1-14 

- 00-1967  IFAC  ■0jHI4 

X  *  2-IFAC 

. . . Y~r"X»l4' . 

2967  PROP  ■  PR0Q4X/V 

1119  *  M! 

. DO  2969  IPAC*Oi 1119 

X  «  2-IFAC 

-  Y  *  X-19 

2969JPR0D  •PROOiX/Y 

1 1 16  ■* 1-16 

-  . 00  2971  IFACmOi  1 1 16 

X  •  2-IFAC 

. .  Y  9  X-I6 

2971  PROD  *  PRD09X/Y 

2974 

III?  *  1-17 

. . 00  29-73  rFACiOilin 

X  ■  2  -  X FAC 

.  Y  •  X-17 

2971  PROO  ■  PR009X/Y 
-2974  60  TO'  2976 

1118  •  1-18 

. DO  2979  1FAC-0jIII8 

X  ■  l-IFAC 

Y  «  X-l 8 

2979  PROD  ■  PR0D4X/Y 
-6076-1F t!9.tO'.2)  CO  TO  2978 
1119  ■  1-19 
DO  2977  IF  AC  ■  0*1119 
X  *  2-IFAC 

Y  •  X-19 

2977  PROD  •  PRQD*X/Y 
-29T6-1F1-1 10,20.2)  GO  TO  2980 
1 1 X 10  ■  1-110 
00  2979  IFAC"Oj  1 1 1 1 0 
X  ■  2-IFAC 

Y  *  X-110 

2979  PROD  •  PROO*X/Y 
-2980  IFdll.EQ.n  GO  TO  2982 
mu  -  i  -  lii 

00  2981  1 F AC  ■  0*11111 
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'3C0. 

SOI. 

-501  r 

501. 


356* 


1  900. 
505. 
-«4v 
SOT. 
"366v 
*69. 


3367 

1561 


-w. 

s \i» 

-Mi¬ 

ni* 

-5t4r. 

515, 


—  +  *  — 

5  IT, 
-~3i8v 
515. 
-5107- 
521. 
51  tr 


t V* ( X * •  CQ. 2 1  GO  TO  3568 
1114  •  1-16 
00  3567  iFAC  *0*1114 
X  ■  2-lFAC 

T-»“Tc«rv“ . . 

PROD  •  prqd*x/v 
1FMB.EQ.2)  60  TO  3570 
1U5  *  1-15 

- oo '3365  lPAC«0*l!13 

X  •  2-lFAC 

- Y-r  X»TS - - - 

3569  PROO  ■  PRQD6X/Y 

3576  IP tt 6.10*11 
1116  ■  1-16 

. 00  3871  lPAC*0*UI6 

X  9  2-lFAC 

- -  T  r  r»Tti - . 

3371  PROD  «  PR0D6X/V 

'11M1T.tQ.2>  GO  TO  3574 
HIT  ■  1-17 
00 3873  1PAC«0j1!17 
X  •  l  -  I  PAG  . 


60  70  1872 


-35T2 


-*«rr - 


513. 
■514  i 
525;. 
516* 
517. 


3571 

3574 


3576 


— 

329. 
-~536r 
531,. 
-  511* 
513. 


5167- 

535. 

-•584'.'" 

537. 

538. 

539. 


~544r 
541. 
54*  .  - 
541. 
-344.- 
545. 


1 

~r 


-566- 

547. 

-548. 

549. 

550. 

551. 


553. 

554. 

555. 

556. 

557. 


559* 
560. 


Y~ir 

prod'*  PR0D4X/V 
IF( I8.8Q.2)  60  TO 
U18  *  1-18 
DO  3875  IPAC*0* X X X 8 
X  ■  I-IFAC 

— — y  *  -x»tt - ;■ . 

1578  PROD  «  PRQD4X/Y  -.-a 

3576  tFM9.6Q.2i  60  TO  3|T8 
1119  ■  1-19 

DO  3577  iFAC  •  0*HI9 
X  *  2-1FAC 

—  - y— i— X»T9  " 

3577  PROD  *.  PROD**/* 

-3378  !FM  10* 68.1)  60  TO  3380 

niio  •  l-lio 

00  3879  1FAC*0j11110 
X  ■  I-IFAC  ,  .  . 

—  - y-TT-x-n-o  ■ - 

3379  PROD  •  PR0D6X/V 
*  IPMil*68.2)  00  TO  3382 

mu  •  i  -  iu 

DO  3381  IFAC  *  0*11111 
X  *  2  -  IFAC 

-Y-r3'  riU  . .  " 

PROO  *  PROD  M/y 
IF( 112.68. 2)  OD  TO  3384 

DO*  3583  1PAC1*I0*11112 
X  •  2  -  IFAC 

- -V'-n-X  -  -til 

3381  PROD  ■  PROD  *  X  /  V 

IP ( 1 13.60.2 )  00  TO  3886 
11113  «  1-113 
DO  1389  iFAC'O.inil 
X  ■  2-tFAC 
•Y~r  X*1 13 
PROD  *  PR0D6X/V 
IF(Il4.60.2)  60  TO  3612 
173 


-3380 


3381 

3962 


3984 


3589 

3564 


l'.  aiViv.lLUt, 


361. 

1 1 1 14  -  1  -  IU 

562. 

DO  3611  I FAC  *  0.11114 

1 

563. 

X  *  2  -  I FAC 

1 

364  4 

V  f  K  •  IU 

1 

365. 

3611  PROD  •  PROO  *  X  /  Y 

366, 

3612  A  •  A  A  PROD 

367. 

RETURN 

365. 

END 

END  FTN  3097  IBANK  430  QBANK  218  COMMON 


A 
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IF  (K.OT.J)  60  TO  6 

F8.S...PB  + 

K  •  K  *  1 


— Jl_PJLj*_EA  [ 1-RA  >**KJ •(RA**  t  I«K5  H 

K  ■  K  *  l 

_ LEilL-n  ItiitQOO 

ioo  y  ■  x-v 


END  FTN  110  IBANK  ZU  DBANK 


IBBKPJLMINTl 
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The  graph  here  Is  of  PA  values  for  NS  =  4  and  IS  =  10,  20,  and  30.  Note 
that  DA  =  28  Is  Impossible,  so  a  value  of  26.01  entered  Into  program  WA  or  WA1 
would  yield  the  proper  result  for  DA  =  30. 


APPENOIX  C 

R  VALUES  TO  NORMALIZE  DISTRIBUTION  OF  THE  ALTERNATIVE  HYPOTHESIS 


NS 

IS 

w 

R 

4 

20 

0.40 

0.500129 

4 

20 

0.30 

0.500553 

4 

20 

0.25 

0.500913 

4 

30 

0.30 

0.500367 

4 

14 

0.25 

0.501313 

4 

10 

0.30 

0,501122 

4 

10 

0.25 

0.501856 

4 

8 

0.25 

0.502339 

3 

60 

0.40 

0.500092 

2 

20 

0.30 

0.501122 

2 

20 

0.25 

0.501856 

2 

10 

0.25 

0.503840 

2 

40 

0.40 

0.500129 

C-l 
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APPENDIX  D 


This  Is  a  comparison  of  the  exact  test  results  of  program  WA  with  the 
Chi-square  results  given  by  the  program  "CROSSTABS"  found  In  the  SPSS,  and/or  a 
program  for  Chi-square  with  three  degrees  of  freedom  found  In  Appendix  E, 
written  by  Dr,  J.  V.  Blowers.  Dr.  Blowers'  program  Is  applicable  any  time 
NS*4.  However  CROSSTABS  Is  only  applicable  when  calculating  PA. 
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EXACT  TEST 
NS  =  4 


CHI-SQUARE 
NS  =  4 
IS  *  20 


IS  =  20 


H  :  10 

10 

10 

10 

H  :  10 

10 

10 

10 

0 

0 

10 

10 

10 

10 

10 

10 

10 

10 

H,.  5 

5 

15 

15 

H. .  5 

5 

15 

15 

‘  15 

15 

5 

5 

(l.t.,  W  =  0.25) 

l 

15 

15 

5 

5 

Observed: 

7 

8 

12 

13 

Observed: 

7 

8 

12 

13 

13 

12 

8 

7 

13 

12 

8 

7 

PA  =  0.1906 
PB  =  0.0917 


PA  *  0.1577  (from  "CROSSTABS") 

PB  =  0.0741  (from  program  by 
J.  V.  Blowers) 


EXACT  TEST 
NS  *  4 
IS  ■  20 


v  10 

10 

10 

10 

10 

Ho 

10 

10 

10 

u  •  * 

Hr 

6 

14 

14 

H1 

14 

Observed ; 


14 


8  8 
12  12 


12 

8 


12 

8 


PA  ■  0  -  3677 
PB  *  0.2827 


CHI-SQUARE 
NS  *  4 
IS  *  20 

1,0  10 
10  10 


6  6 
14  14 


Obt«rv«d: 


8 


12 


PA  *  0.3618 
PB  *  0.2828 
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EXACT  TEST 
NS  -  4 
IS  »  20 


CHI-SQUARE 
NS  »  4 
IS  -  20 


H  • 

V 

10 

10 

10 

U  .  10 

V 

10 

10 

10 

10 

10 

10 

10 

10 

10 

10 

10 

hi!  8 

8 

12 

12 

•V  8 

8 

12 

12 

12 

12 

8 

8 

12 

12 

8 

8 

Obstrvvd: 

9 

9 

11 

11 

Obstrvtd: 

9 

9 

11 

11 

11 

9 

9 

11 

11 

9 

PA  ■  0.8817 
PB  «  0.8385 


PA  ■  0.8495 
PB  >  0.8415 


0-4 


.182 


EXACT  TEST 

CHI-SQUARE 

NS  ■  4 

NS  *  4 

IS  *  10 

IS  =  10 

H0t  5  5  5 

5 

u  .  5  5 

V 

5  5 

5  5  5 

5 

5  5 

5  5 

H..  2‘*  2'5 

7.5 

7.5 

u  .  2.5  2.5 

Hr 

7.5  7.5 

7.6  7.5 

2.5 

2.5 

7.5  7.5 

2.5  2.5 

3 

Observed: 

7 

4  6 

6  4 

7 

3 

Observed: 

3  4  6 

7  6  4 

PA  »  0.3305 

PA  «  0.2615 

PB  ■  0.6196 

PB  »  0.4459 

4 

Observed: 

6 

4  6 

6  4 

6 

4 

Observed: 

4  4  6 
6  6  4 

7 

3 


6 

4 


PA  -  0.7269 
PB  *  0.2912 


PA  »  0.6594 
PB  ■  0.1871 
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EXACT  TEST 
NS  *  4 
IS  *  30 


V  15 

15 

15 

15 

15 

15 

15 

15 

Hr  9 

9 

21 

21 

21 

21 

9 

9 

Observed: 

13 

14 

16 

17 

17 

16 

14 

13 

PA  *  0.7626 
PB  *  0.0033 


Observed:  ^  H  13 

19  19  11  11 

PA  *  0.0375 
PB  -  0,4600 


CHI-SQUARE 
NS  b  4 
IS  =  30 


U  .  15 
V 

15 

15 

15 

15 

15 

15 

15 

H1J  9 

9 

21 

21 

21 

21 

9 

9 

Observed: 

13 

14 

16 

17 

17 

16 

14 

13 

PA  ■  0.7212 
PB  *  0.0046 


Observed:  ^  H  13  19 

19  19  11  U 

PA  *  0.0362 
PB  -  0.4682 


0-6 
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APPENDIX  E 


Computer  program  far  Chl-Squara  with  thraa  digraas  of  fratdom  (Blowara). 
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9FTN*S  D1K , J VBC S3* KEK 

FTN  10K1  01/05/62-11158(11# > 


PROGRAM  JVB 
DIMENSION  MIliHII) 

«0*WRITEr5/n - - 

WRITE  (6/ *  A ), 

A A ' FORMAT! )X#*AAAAAAAB6BBBBBCCCCCCC0DDDdD0E EE EEEEFFFFFFP* 
l  •OOGOSGOHHHHHHHt) 

•r  FORHiTrmiiiifUT“DATA-iy'iio«i#PT.*'  Format* i - - 

READ(l»t)(Am;iM#lf 


IJII  41  T>  t] 


00  1000  M*l 


l*l*»f  ULLT  l.UHHLIiHI  11  1*  1IIL_l.lt  Viriiittl 


TH-CHISKSAH) 

- FR1  m*7l-OTI  ATtt  ir-TH - — - - - 

10  FORMAT! IX#  ICHMOUARE'I, FIB. It t  SIONIFICANCI  -  ifFll'.SI 


FUNCT1UN-CHISICXN1 

H-XN/lOOO. 

-CH1SB-0.*  —  - 

.  DO  1.  1*0*1000 


IP'l  I*NI»  (I/2)*t)C*A 


[ij|n  rirn^  n n 


FUNCTION  FIX! 

F*IQAT  ( X'/I*  /S  •  lAlBT|ABt5BT79)*IXF  ( vXyfi 


tNO-FtN-rtt-lOANK-HI-frtANX-- 


BBRKFT  PRINT* 


APPENDIX  F 


Th1»  1*  *  comparison  of  WB1  and  WC  program  results  daslgntd  to  show  whan 
tha  null  hypothasas  In  aach  cast  ara  equivalent.  Tha  last  paga  of  this 
appendix  provldas  RA  valuas  that  causa  approximate  equivalency  for  soma  NS  and 
NZ  (or  DA)  conditions.  In  projecting  RA  and  RB  valuas  to  usa  whan  NS  valuas 
ara  too  large  to  usa  program  W8I,  a  high  estimate  will  ba  bast  as  that  will 
result  In  a  PA  or  PB  value  that  Is  smaller  than  tha  result  should  be.  and  will 
help  reject  doubtful  hypothasas. 
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NS  *  8 
DA  -  NZ  -  2 

DA  *  NZ  *  4 

DA  *  NZ  ■  6 


WC 

0.9791  whan  RA  *  0.38 
(0.9813  whan  RA  ■  0.37) 

0.7023  whan  RA  *  0.49 

0.1792  Whan  RA  *  0.60 


DA  -  NZ  -  8 


0.0053  whan  RA  ■  0.73 


wc 

NS  -  12 
DA  a  NZ  a  12 

0.9985  when  RA  a  0.34 


DA  *  NZ  ■  4 


0.9490  when  RA  -  0.42 


DA  «  NZ  ■  6 

0.6748  when  RA  -  0.49 
(0.6562  when  RA  *  0.50) 

DA  »  NZ  »  8 

0.2232  when  RA  »  0.57 
(0.2390  when  RA  *  0.56) 
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Addendum: 

In  Bradley,  Distribution-Free  Statistical  Tests,  pages  237-242,  the 
Brown-Mood  Multi-Sample  Median  Test  turns  out  to  be  "  .  .  .a  generalization  to 
the  multi-sample  (and  multi-population)  case  of  the  two-sample  Westenberg-Mood 
Median  Test  .  .  This  Is  part  of  what  has  been  suggested  here  although  the 
Interquartile  range  Is  of  more  Interest  In  thl.s  study  since  there  seems  to  be  a 
greater  selection  of  median  oriented  tests.* 

Some  modifications  (corrections)  can  be  made  to  the  chi-square  test  to  make 
It  a  better  approximation.,  It  has  been  state^  (Bradley,  page  239)  that 

multiplying  .this  X*  test  statistic  by  (n-l)/n  will  improve  the  approximation 
(Mood)  (where  n  Is  the  combined  sample  size)  arid  this  appears  to  be  true  In  the 
examples  g 1  yen  here  as  applied  to  the  null  hypothesis,  but  not  the  alternative 
hypotheses,  (Bradley's  book  does  not  specifically  address  such  alternative 
hypotheses  directly,  but  Instead  deals  with  asymptotic  relative  efficiencies 
(AREs) . ) 


*In  reference  4,  It  was  noted  that  the  Kruskal-WalHs  Test  Indicated  median.,: 
K.P.  times  from  questionnaires  administered  to  various  posts  were  probably 
Identical  when  It  was  obvious  that  the  dispersions  were  different,  That  is  why 
dispersion  was  emphasized  In  this  paper, 


A  DATA  BASED  RANDOM  NUMBER  GENERATOR  FOR  A  MULTIVARIATE  DISTRIBUTION* 

Janes  R.  Thompson  Rice  University 

Malcolm  S.  Taylor  USA  Ballistic  Research  Laboratory 

ABSTRACT.  Let  X  be  a  k-dimensional  random  variable  serving  as  input  for  a 
system  with  output  Y  (not  necessarily  of  dimension  k) .  Given  X,  an  outcome 
Y  or  a  distribution  of  outcomes  G (Y | X'j  may  be  obtained  either  explicitly  or 
implicitly.  We  consider  here  the  situation  in  which  we  have  a  real  world 
data  set  and  a  means  of  simulating  an  outcome  Y.  A  method  for 

,i  1  ■  i 

empirical  random  number  generation  based  on  the  sample  of  observations  of 
the  random  variable  X  without  estimating  the  underlying  density  Is  discussed. 

INTRODUCTION.  The  manner  of  dealing  with  multivariate  data  depends  upon  the 
application  at  hand.  For  example,  let  vis  suppose  that  is  a  sample 

of  sice  n  of  a  k-dimensional  random  variable.  We  may  be  interested  simply 
in  estimating  the  mean  u.  In  such  a  case,  we  may  complete  our  task  by  com¬ 
puting  the  sample  mean  X.  If  we  are  interested  in  the  interrelationships 
between  the  various  vector  components,  we  may  find  it  desirable  to  compute 
the  sample  covariance  matrix  5. 

At  a  greater  level  of  complexity,  we  may  be  required  to  estimate  the 
density  of  X  nonparametricelly  [1,3].  Here,  the  representational  difficulties 
are  substantial-—  particularly  for  k  >  2,  where  our  3-dimensional  intuitions 
are  inadequate  for  graphing  the  density  even  if  we  knew  it  precisely  on  a 
discrete  mesh.  Indeed,  it  would  appear  that  for  increasing  dimensionality, 
our  estimation  theoretic  difficulties  pale  in  comparison  to  those  of  repre¬ 
sentation. 


This  research  was  supported  in  part  by  ARO  Contract  DAAG-29-82-K-0014  at 
Rice  University. 
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plUflpflfMs.  HUM  Mi nim 


Suppose  we  are  given,  for  example,  the  task  of  estimating  the  density 
f  at  a  point  XQ  in  k-space,  based  on  a  sample  of  size  n.  The  naive  nearest 
neighbor  estimator 

t(V  '  n  Vk(X0Jd(Jt0Tp5T  • 

where  d(XQ,p)  is  the  Euclidean  distance  from  XQ  to  the  pth  nearest  neighbor 
and  Vjj(X0#d(X  ,p))  is  the  volume  of  the  k-sphere  centered  at  XQ  with  radius 
d(X0«p)«  is  likely  to  be  quite  satisfactory.  But  a  problem  occurs  when  we 
are  asked  for  a  usable  summary  of  the  unknown  density  over  the  space  of  non- 
negligible  mass.  If  we  know  the  functional  form  of  the  density  f(X;B), 
then  we  have  a  relatively  easy  task;---  the  estimation  of  9.  But  in  the 
highly  ubiquitous  nonparametric  situation,  in  which  we  do  not  know  the  func- 

tional  form  of  f,  we  are  not  so  fortunate.  We  might  decide,  for  example,  to 

*  k 

tabulate  f  on  a  mesh  of  size  20  in  each  dimension.  This  would  require  20 

pointwlse  estimations  of  £---  a  tedious  but  manageable  task.  But  how  shall 

we  scan  this  k-dlmensional  table  to  obtain  a  useful  feel  for  the  density? 

Other  approaches,  clearly  are  required.  One  of  these  is  discussed  in  [2]. 

There  are,  happily,  cases  in  which  the  density  representational  diffi¬ 
culties  may  be  sidestepped  when  coping  nonparametrically  with  data  sets  in 
higher  dimensions.  For  example,  let  us  suppose  the  k-dimensional  random 
variable  X  is  on  input  into  a  system  with  output  V  (of  whatever  dimension) . 
Given  X,  an  outcome  Y  or  a  distribution  of  outcomes  G(Y|x)  is  obtained 
either  explicitly  or  implicitly  through  an  output  data  set.  Let  us  suppose 
these  outcomes  fall  into  six  categories:  Very  Good,  Good,  Fair,  Poor,  Very 
Bad,  Catastrophically  Bad.  Suppose  further  that  these  sets  are  well-defined 
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in  the  Y-space.  We  are  given  a  real  world  data  set 
means  of  simulating  an  outcome  Y  given  the  input  X.  We  wish  to  determine 
the  probability  of  arriving  in  each  of  the  six  category  sets. 


One  way  to  achieve  this  result  might  be,  simply,  to  sample  from  the  n 
data  points  In  many  cases  this  will  prove  quite  satisfactory. 

But  let  us  suppose  that  "Catastrophically  Bad"  happens  for  Y  >  10, 


where  Y  -  1/  T  x? 

•'  i«l  1 


with  X  ■  (x1,x2,x3,x4) . 


then,  if |  the  Xj'a  are,  (unbeknownst  to  us,  but  in  actuality)  independently 
distributed  as  N(0,1),  the  chance  of  a  "Catastrophically  Bad"  event  is 
.0012.  Let  us  suppose  the  size  (n)  of  our  data  Set  is  100.  The  chance  of 
none  of  these  observations  being  in  the  "Catastrophically  Bad"  region  is 
,887.  So,  a  simulation  which  used  only  the  100  data  points  would,  with 
probability  .887,  give  us  the  information  that  "Catastrophically  Bad" 
occured  with  zero  probability.  We  need  to  avoid  this  pitfall. 


One  procedure  would  be  to  estimate  the  density  of  X  nonparametrically 
and  then  build  a  random  number  generator  using  the  density.  Such  a  scheme 
would  run  into  the  representational  difficulties  mentioned  above,  We  can 
be  much  more  efficient. 


THE  ALGORITHM.  Let  us  consider  the  following  situation:  We  have  a  random 
sample  of  size  n  from  a  multivariate  distribution  of  dimension  k, 

and  we  want  to  generatt  pseudorandom  vectors  from  the  underlying,  but  unknown, 
distribution  that  gave  rise  to  the  random  sample.  Since  we  do  not  know, 
and  usually  will  never  know,  the  form  of  this  distribution,  our  attack 
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should  be  empirical.  We  shall  endeavor  to  see  to  it  that  our  pseudorandom 
vectors  look  very  much  like  those  in  the  original  data  set.  In  so  doing,  we 
will  maintain  the  essential  structural  Integrity  of  the  problem. 


We  now  direct  our  attention  to  the  mechanics  of  the  algorithm.  After 
carrying  out  a  rough  rescaling  to  account  for  differing  variances  that  may 
exist  among  the  k  variates,  we  select  at  random  one  of  the  n  data  points, 
say  X^,  from  the  data  base  and  then  proceed  to  determine  its  m-1  nearest 
neighbors.  The  nearest  neighbors  are  determined  under  the  ordinary  Euclidean 
metric  and  the  value  of  m  will  depend  upon  the  sample  size  n,  the  character¬ 
istics  of  the  data,  and  can  best  be  determined  after  perusal  of  the  data. 

A  conservative  estimate  would  be  to  choose  m  ■  n/20. 


The  vectors  are  now  coded  about  the  sample  mean  X  »  l/m  E  X, 

to  yield  (xj>  -  <Xj  •  and  an  independent  random  sample  of  size  m  is 


generated  from  the  uniform  distribution  U(l/m 
Now  the  linear  combination 


.  ,  l/m  .  ). 


m 


i .  v 


4-1 


int 


is  formed,  where  is  the  random  sample  from  the  U(l/m  -  /•",  l/m  +  /•'). 

Finally  the  translation 


X  ■  X'  +  X 

restores  the  relative  magnitude,  and  X  is  a  pseudorandom  vector  which  we 
propose  to  be  representative  of  the  multivariate  distribution  that  provided 
th«  . 
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To  obtain  the  next  pseudorandom  vector  we  randomly  select  another  of 
the  n  data  points  and  proceed  as  above. 

We  will  now  attempt  to  motivate  the  algorithm  by  considering  the  mathe¬ 
matics  that  suggests  the  mechanics  that  we  have  just  outlined.  Consider 
the  distribution  of  and  its  m-1  nearest  neighbors: 

{(x1£(x2£,  ...,xkA)  l<*t  us  suppose  that  this  "truncated  set" 

of  random  observations  has  mean  vector  u  and  covariance  matrix  o,  Let 
SCi  be  an  independent  random  sample  from  the  uniform  distribution 
U(l/m  -  /%  1/m  ♦  /•*  ).  Then,  B(u£)  ■  1/m,  Var(u£)  ■  (m-l)/m2,  and 
CovC^,  Uj)  ■  0,  for  i  +  j. 

Forming  the  linear  combination 

2  "  ? 

1*1  1  1 

we  have,  for  tho  rth  component  «r  ■  UjXrl  +  u2xr2  ♦  ...  +  u^^,  the  following 
relations 

B(2r)  ■  m  •  1/m  •  ur  ■  ur, 

Var(tr)  ■  oj  ♦  (m-l)/m  •  yj, 

Cov(tr,*iD  ■  or8  ♦  Cm-l)/m  •  yry#. 

Clearly,  if  tho  mean  vector  of  X  was  m  (0,0,  ...,0)',  then  the  mean  vector 
and  covariance  matrix  of  Z  would  be  identical  to  those  of  X.  In  the  less 
Idealized  situation  with  which  we  are  confronted,  the  translation  to  the 
sample  mean  of  the  nearest  neighbor  cloud  should  result  in  the  pseudoobserva¬ 
tion  having  very  nearly  the  same  mean  and  covariance  structure  as  that  of  the 


201 


(truncated)  distribution  of  the  points  in  the  nearest  neighbor  cloud,  a  con¬ 
jecture  borne  out  in  many  actual  cases  that  have  been  considered,  For  m 
moderately  large,  our  algorithm  essentially  samples  from  n  Gaussian  distribu¬ 
tions  with  the  means  and  covariance  matrices  corresponding  to  those  of  the 
n  m  nearest  neighbor  clouds. 


EXAMPLES.  For  a  substantial  test  case,  we  considered  a  mixture  of  three  bi¬ 
variate  normal  distributions.  The  first  (N^)  has  mean  vector  (*2)  and  covariance 

1  -1/2  -2 
matrix  (_jy2  i  );  the  second  (N2)  has  mean  vector  (  3)  and  covariance  matrix 

1  1/2  2 
^1/2  1  and  the  third  (Nj)  has  mean  vector  (3y2)  and  covariance  matrix 

^1/10  1i10)*  The  corresponding  mixing  scalars  are  ■  1/2,  a2  ■  1/3,  and 
a3  ■  1/6,  respectively,  Representative  contours  of  equal  density  are  illus¬ 
trated  in  Figure  1.  To  establish  a  data  base,  a  sample  of  eighty-five  points 
was  generated  from  this  distribution  via  Monte  Carlo  simulation;  a  sample  of 
eighty-five  pseudorandom  values  was  then  produced  by  the  algorithm,  and  the 
combined  sample  is  shown  in  Figure  2. 


Notice  that  the  structure  of  the  data  is  maintained  in  that  the  modes 
are  preserved;  the  algorithm  has  not  attempted  to  fill  in  gaps  where  gaps 
belong;  the  algorithm  has,  however,  generated  some  points  outside  the  boundary 
of  the  convex  hull  of  the  data  base,  all  of  which  are  desirable  properties. 
These  observations  lend  credence  to  the  term  "structural  integrity"  mentioned 
previously. 


An  application  of  the  algorithm  to  a  real  world  data  set  is  summarized 
in  Figures  3  and  4.  In  Figure  3,  a  two-dimensional  marginal  of  a  set  of  973 
four-dimensional  behind  armor  debris  measurements  is  portrayed;  in  Figure  4, 
973  simulated  data  points  produced  by  our  procedure.  Once  again,  the  salient 
features  of  the  data  set  are  preserved. 
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Fig.  2.  Combi ntd  samplt:  Data  bast  and  Psaudoobscrvatlom. 
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Fig.  4.  Simulated  behind  armor  debris. 
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CONCLUSIONS.  We  have  demonstrated  a  means  of  empirical  random  number  genera¬ 
tion  based  on  a  sample  of  observations  of  a  random  variable  X.  No  esti¬ 
mation  of  the  underlying  density  is  required.  And,  because  of  the  local 
nature  of  the  generation  scheme,  it  is  essentially  free  of  assumptions  on 
the  underlying  density  of  X.  Naturally,  any  attempt  to  use  this  algorithm 
for  generating  bona  fide  new  observations  using  the  computer  rather  than 
producing  real  world  data  would  be  unwise.  Rather,  the  algorithm  operates 
somewhat  like  a  smooth  interpolator—  highly  dependent  on  the  quality  of  the 
data  points  on  which  it  is  based.  It  gives  us  a  means  of  avoiding  nonrobust 
conclusions  due  to  "holes"  in  the  data  set  at  important  points  of  the  simula¬ 
tion  model. 
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I.  INTRODUCTION. 

A.  During  a  routine  error  propagation  analysis,  a  continuous  function 
with  piece-wise  continuous  first  derivatives  was  encountered.  A  couple 

of  Interesting  effects  were  observed.  These  are  described  In  this  paper. 

The  magnitude  of  these  effects  are  quite  small  but  they  are  quite  distinctive. 
Their  explanation  Is  quite  simple  but  they  are  observed  so  rarely  that 
their  explanations  are  not  well  known. 

B.  The  source  of  the  particular  equation  was  nuclear  radiation 

testing.  In  the  equipment  under  study,  the  radiation  source  and  detector 
are  In  air  but  may  be  separated  by  steel.  The  derivation  of  the  function 
Is  outlined  In  Figure  1;  I  Is  the  Intensity,  C  Is  the  number  of  photons 
emitted  per  second  by  the  source,  D  Is  the  distance  that  the  radiation  ■ 

penetrates  the  steel,  and  p  is  the  attenuation  coefficient  of  the  steel. 

When  the  detector  distance  R  Is  twice  the  source  parameter  A,  the  distance 
D  Is  zero;  thus,  R  ■  2A  Is  the  break-point  between  the  two  functions 
describing  the  Intensity  I.  This  piece-wise  continuous  function  for  I 

Is  plotted  In  Figure  2  when  the  product  pB  Is  taken  to  be  unity. 

II.  ROUTINE  ERROR  ANALYSIS. 

A.  The  routine  random  error  analysis  of  the  equation  Is  outlined  In 
Figure  3.  Only  the  standard  deviation  In  R  Is  considered  In  the  propa¬ 
gation  of  error  to  I  because  the  random  error  Instead  of  the  total  error 
is  the  Item  of  Interest  and  the  bias  error  receives  the  contributions 
from  the  standard  deviations  In  C,  A,  B,  and  p.  The  result  Is  presented 
graphically  In  Figure  4  with  numerical  values  of  one  for  uB,  eight 
centimeters  for  A,  and  two  millimeters  for  6R. 
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B.  The  discontinuity  at  the  break-point  (at  R  *  2A  ■  16  cm)  results 
mathematically  from  the  Inequally  of  the  absolute  value  of  the  slope  of 
1  versus  R  on  the  two  sides  of  the  break-point.  An  obvious  question  Is: 
"Is  there  a  standard  error  propagation  procedure  to  eliminate  this 
discontinuity  and  obtain  a  better  estimate  of  the  error  at  the  break¬ 
point?"  Perhaps,  the  answer  Is  that  the  question  Isn't  relevant  since 
simulation  can  be  used  Instead  of  error  propagation. 

III.  SIMULATION. 

A.  The  estimation  of  the  error  at  the  break-point  can  be  done  by 
simulation.  All  simulations  presented  In  following  Figures  will  be  based 
on  500  random-normal  samples  for  each  plotted  point  of  R.  The  standard 
deviation  of  R  Is  retained  at  two  millimeters. 

B.  The  first  effect  noted  In  simulation  Is  the  appearance  of  horns 
at  the  break-point  of  graphs  relating  fractional  standard  deviations 
from  simulations  to  the  radius.  Figure  5  presents  a  simulation  that  Is 
too  course  In  R  values  to  tell  how  the  standard  deviation  behaves  as  R 
passes  through  the  break-point.  Figure  6  presents  a  simulation  of  the 
fractional  standard  deviation  of  R  values  near  the  break-point.  A  horn 
appears  at  the  break-point. 

C.  The  second  effect  noted  In  simulation  Is  the  appearance  of  dual 
valleys  In  the  graphs  of  fractional  standard  deviations.  This  occurs 
when  the  Intensity  function  Is  adjusted  to  make  the  absolute  value  of 
dl/dR  continuous  at  the  break-point.  As  Figure  7  and  Figure  8  Indicate 
the  attentuatlon  coefficient  can  be  set  to  make  both  |dI/dR|  and  $1/1 
from  error  propagation  continuous.  Even  though  Figure  9  shows  that 

61 /I  Is  Indeed  continuous  and  Figure  10  shows  that  the  simulation  of 
Oj/I  generally  follows  6I/I,  the  detailed  view  of  Figure  11  reveals 

an  Irregularity.  At  first  glance,  there  appears  to  be  a  horn  at  the 
break-point,  but  closer  examination  shows  that  cr^/I  at  the  break-point 

has  the  expected  value  while  Oj/I  Is  depressed  on  both  sides  of  the 

break-point. 

0.  A  deep,  single  valley  Is  obtained  on  the  simulation  of  Oj/I 

when  the  sharp  point  at  the  break-point  Is  rounded  with  a  smoothing 
factor.  Figure  12  shows  the  smoothing  factor  and  parameters  used.  The 
resulting  Intensity  curve  Is  shown  In  Figure  13.  The  simulation  of 
Oj/I  versus  R  yields  a  deep  valley  shown  In  Figure  14:  Figure  15  shows 

that  the  bottom  of  this  valley  Is  flat. 
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IV,  EXPLANATION. 


A,  The  origin  of  these  Irregularities  Is  clearest  when  attention  is 
focused  on  the  valleys  In  Oj/I.  Probable  error  bars  of  magnitude 

(2) (2/3) (crj )  ■  2  2/3  millimeters  may  be  used  as  a  tool  to  aid  In  understanding 

the  values  of  Cj/1. 

B.  The  easiest  irregularity  to  describe  Is  the  deep  valley  of 
Figure  14  and  Figure  15.  Figure  16  shows  an  enlarged  view  of  the  rounded, 
symmetric  point  on  I{R)  versus  R.  The  probable  error  bars  on  R  are  > 
used  to  define  the  pseudo-error  of  I.  This  equation  was  used  to  calculate 
the  curve  on  Figure  17.  Figure  17  Is  in  good  qualitative  agreement  with 
Figure  14  and  Figure  16. 

C,  Next,  the  Irregularity  of  Figure  11  may  be  described.  Figure  18 
shows  an  enlarged  view  of  the  symmetric  point  on  I (R)  versus  R.  The 
same  pseudo-error  function  that  was  used  to  describe  the  valley  of  Figure 
14  and  Figure  16  also  generates  the  curve  of  Figure  19.  This  curve 
qualitatively  describes  the  to-shaped  dual  valleys  of  Figure  11. 

D.  Finally,  the  horn  of  Figure  6  Is  described  by  the  same  technique. 

The  result,  shown  on  Figure  20,  not  only  reproduces  tne  horn  but  shows 
that  sides  of  horn  are  depressed  Instead  of  the  point  being  raised. 
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ABSTRACT 

An  Inferential  procedure  Is  presented  which  provides 
confidence  Intervals  for  a  future  reliability  parameter 
when  reliability  growth  testing  Is  only  partially  completed. 
Hypothesis  tests  based  on  this  method  are  uniformly  most 
powerful  unbiased.  These  results  are  applicable  If  (1)  the 
system  failure  rate  can  be  modeled  as  the  Intensity  function 
of  a  Wei  bull  process  and  (2)  efforts  to  Improve  reliability 
are  assumed  to  continue  at  a  steady  rate  throughout  the  Inter 
venlng  period  of  testing.  The  usefulness  of  this  methodology 
Is  Illustrated  by  evaluating  the  risk  of  not  reaching  some 
future  reliability  milestone.  If  such  risk  Is  unacceptably 
high,  program  management  may  have  time  to  Identify  problem 
areas  and  take  corrective  action  before  testing  has  ended. 

As  a  consequence,  a  more  reliable  system  may  be  developed 
without  incurring  overruns  In  the  scheduling  or  cost  of  the 
development  program. 


1.  INTRODUCTION 


Reliability  growth  management  Is  a  critical  function  In  the  development 
programs  of  major  defense  systems. 1  It  consists  of  planning,  monitoring,  and 
controlling  the  growth  of  reliability  parameters  throughout  system  development 
In  order  to  achieve  the  reliability  milestones  for  each  test  phase  and  for  the, 
overall  program.  A  key  factor  in  this  process  ts  the  ability  to  assess  the 
risk  of  not  meeting  a  reliability  requirement  and  to  make  such  assessment  at 
an  early  stage  in  the  current  test  phase.  If  this  risk  is  unacceptably  high, 
the  program  manager  may  then  have  an  opportunity  to  take  remedtal  action 
before  test  time  or  other  program  resources  are  exhausted.  The  risks  of  fall¬ 
ing  to  achieve  program  goals  or  contractual  requirements  can  therefore  be 
minimized.  Instead  of  having  to  react  to  program  shortcomings  after  the  fact, 
management  can  exert  positive  control  over  the  growth  process  to  accomplish 
reliability  objectives. 

Reference  1  (pp.  10,  23,  28,  64-66,  76-78}  discusses  the  use  of  reliability 
growth  models  to  project  reliability  estimates  beyond  the  present  test  time  to 
some  future  time,  such  as  the  end  of  the  currant  tost  phase.  These  projections 
are  valid  only  if  test  conditions  remain  relatively  constant  and  the  develop¬ 
ment  effort  continues  at  its  previous  level.  The  projected  reliability 
estimates  are  compared  with  future  milestones  In  order  to  assess  whether  the 
reliability  enhancement  program  is  likely  to  reach  a  successful  conclusion. 

One  of  the  problems  with  assessing  a  program  by  this  method  is  how  to 
evaluate  the  accuracy  of  the  reliability  projections.  Such  projections  are 
only  point  estimates  and  do  not  reflect  the  uncertainties  that  accompany 
random  sampling  from  a  probabilistic  model.  In  this  paper  we  show  how  to 
quantify  these  uncertainties  when  the  Weibull  process  is  used  to  model  and 
forecast  reliability  growth.  The  result  is  an  objective  appraisal  of  current 
program  risks,  and  this  appraisal  can  be  factored  into  those  management 
decisions  which  may  Impact  on  future  reliability  parameters. 

The  Weibull  process  model  has  been  successfully  applied  to  the  reliability 
test  results  of  many  complex  defense  systems.  It  is  Introduced  In  Section  2 
In  a  parametric  form  that  is  especially  suited  to  the  problem  of  forecasting. 
The  basic  features  of  this  model  are  described  in  Appendix  C  of  Reference  1, 
which  includes  confidence  interval  procedures  for  the  reliability  of  the 
current  system  configuration.  (See  also  References  2  and  3.)  The  theory 
developed  in  Section  3  extends  these  latter  results  to  provide  inferential 


^Department  of  Defense,  Reliability  Growth  Management.  Military  Handbook  189, 
Naval  Publications  and  Forms  Center,  Philadelphia,  PA,  February  1981. 

2Bain,  L.  J.  and  M.  Engel hardt,  "Inferences  on  the  Parameters  and  Current 
System  Reliability  for  a  Time  Truncated  Weibull  Process,"  Technometrics. 

Vol.  22,  pp.  421-426,  August  1980. 

3 Crow,  L.  H.,  Confidence  Interval  Procedures  for  Reliability  Growth  Analysis, 
Technical  Report  No.  T9Ti  U  S  Army  Hateriel  Systems  Analysis  Activity, 
Aberdeen  Proving  Ground,  MD,  June  1977. 
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procedures  for  future  reliability  levels.  These  procedures  are  Illustrated 
In  Section  4,  where  confidence  intervals  are  obtained  for  the  reliability  to 
be  achieved  at  future  points  In  a  test  phase  which  Is  still  in  progress.  Also 
obtained  by  an  equivalent  technique  Is  the  risk  of  not  achieving  a  certain 
reliability  level  at  the  end  of  the  test  phase. 

2.  SPECIFICATION  OF  MODEL 

Consider  a  reliability  growth  test  phase  which  has  been  underway  for  T 
units  of  testing.  We  shall  hereinafter  regard  these  test  units  as  time, 
although  they  could  equally  well  represent  other  units  such  as  distance. 
Suppose  that  the  test  phase  began  at  time  0,  but  Is  planned  to  continue  for 
an  additional  S  units  of  testing  till  test  time  T+S,  at  which  point  the  system 
configuration  will  have  failure  rate  R.  Our  objective  Is  to  make  Inferences 
about  the  parameter  R. 

A  Welbull  process  Is  a  nonhomogeneous  Poisson  process  with  an  Intensity 
function  that  can  be  expressed  as  a  multiple  of  some  power  of  the  test  time. 
For  the  particular  test  phase  described  above,  an  Intensity  function  of  the 
appropriate  parametric  form  Is 

r(t)  -  RCt/CT+S)]*"1,  (1) 

where  R>0,  B>0,  and  0<t<T+S.  As  shown  In  Figure  1,  the  function  r(t)  models 
the  failure  rate  of  the  system  configuration  as  It  changes  over  a  reliability 
growth  test  phase  of  length  T+S,  and  the  failure  rate  at  the  end  of  the  (as 
yet  uncompleted)  test  phase  Is  given  by  r(T+S)  ■  R. 

The  failure  rate  model  In  Figure  1  shows  a  decreasing  trend  during  future 
testing  from  time  T  to  time  T+S.  This  trend  reflects  our  previously  stated 
Intention  to  continue  reliability  Improvements  throughout  this  period.  The 
case  In  which  reliability  Is  constant  from  T  to  T+S  is  treated  In  Reference  4. 

According  to  the  scenario  of  this  paper,  test  results  are  available  for 
the  test  period  from  time  0  to  the  (current)  time  T,  but  the  system  testing 
from  time  T  to  time  T+S  has  not  yet  been  accomplished.  Let  N  be  the  number 
of  failures  that  occur  before  time  T  and  T-j ,  . ..,  the  observed  failure 

times  (0<T1  <  ...  <TN<T).  Then  the  Poisson  process  with  intensity  function 

r(t)  has  a  sample  function  density  given  by 

. V"’  *' . *»’ 


"Miller,  577  "Efficient  Methods  for  Assessing  Reliability,"  Proceedings  of  the 
Nineteenth  Annual  U  S  Army  Operations  Research  Symposium.  Part  III,  pp.  33-42, 
October  l&ftT 
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3.  DERIVATION  OF  RESULTS 
3.1  Point  Estimators 


The  Welbull  process  model  Is  used  In  applications  where  Pr(N  *  0)  Is 
quite  small,  and  therefore  the  likelihood  expression  In  Equation  (2.2)  can  be 
maximized  to  obtain  point  estimators  for  8  and  R  as  follows: 

N 

8  -  N/  l  1n(T/T,),  (3) 

1-1  1 

ft 

R  -  N8QM/T.  (4) 

As  would  be  expected,  the  expression  In  (3)  Is  Identical  to  the  estimator  for 
6  In  Reference  2  (Equation  (4)).  The  projected  mean  time  between  failures 
(MTBF)  for  the  system  configuration  at  the  end  of  the  test  phase  (time  T+S)  Is 

estimated  by  R"1. 

The  point  estimators  In  Equations  (3)  and  (4)  are  convenient  because 
of  their  simplicity,  but  were  obtained  without  conditioning  formally  on  the 
event  N>0.  As  a  practical  matter,  Inferences  on  the  two-parameter  Welbull 
process  are  possible  only  when  N>0,  and  we  shall  condition  on  this  event  In 
the  sequel  without  further  mention. 

3.2  Reduction  of  the  Parameter  Space 

N 

Let  V  ■  ^ln[(T+S)/Tj],  and  observe  from  Equation  (2)  that  V  Is  a 

sufficient  statistic  for  B.  It  follows  from  Reference  6  (pp.  134-140)  that 
uniformly  most  powerful  unbiased  (UMPU)  hypothesis  tests  on  the  future  failure 
rate  R  can  be  constructed  by  utilizing  the  conditional  distribution  of  N  given 
V-v.  To  obtain  this  distribution,  we  begin  by  determining  the  conditional 
distribution  of  V  given  N-n. 

Given  N-n,  the  random  variables  T^ *  ....  Tn  are  distributed  as  the 

order  statistics  from  n  Independent  distributions  with  cumulative  distribution 
function 


t  T 

F(t)  ■  /  r(x)dx/J  r(x)dx 
0  0 

-  (t/T)$,  (5) 


6Lehmann,  E.  L.,  Testing  Statistical  Hypotheses.  John  Wiley  and  Sons,  New  York, 
NY ,  1959. 


237 


nrpzTT'  c snsw^e  v'  •IT Vt.VV—T?!® 


ripr^-api” 


where  0<t<T.  Let  X  be  a  random  variable  with  distribution  function  F.  Thtn  ,? 
straightforward  calculation  shows  that  the  random  variable  ln[(T+S)/X]  is 
distributed  over  the  Interval  (lnQ,«)  according  to 

Pr{lnC(T+S)/X]  <  y}  ■  1-expt-(y-lnQ)B],  (61 

where  1nQ<y<»,  This  latter  function  Is  a  two-parameter  exponential  distri¬ 
bution  function  on  the  Interval  (IhQ,*).  The  conditional  distribution  of  V 
given  N*n  Is  therefore  the  sum  of  n  such  distributions »  all  Independent,  and 
consequently  Is  a  three-parameter  gamma  distribution  with  density  function 

Vv|h)  ' 

■  en(v-nlnq),"'»xp[-B(v-n!nQ)]/(n-l ) I ,  (7) 

where  nlnQ<v<«. 

The  random  variable  N  Is  Poisson  distributed  with  mean  value 
es(RT/3),Q1‘0,  so  that  (conditional  on,  N>0) 

Pr(N»n)  ■  [l-tixp(-e)]"1enexp(-e)/ni,  (8) 

n  ■  1,  2,  ...  ,  Thus  the  joint  density  function  of  V  and  N  Is 

fv>N(v.n)  ■  fVjN(v|n)  Pr(N»n) 


where  n  -  1 ,  2,  . . .  and  n1nQ<v<», 

In  the  case  S»0  (forecasting  zero  time  Into  the  future),  we  see  that 
lnQ-0  and  that  the  results  In  this  paper  generalize  certain  results  In  [3] 
and  [2]  on  Inferences  for  current  system  reliability.  In  the  case  S>0,  the 
above  Inequality  n1nQ<v<*  Implies  that  N  has  finite  support,  given  V»v: 

Pr(0<N<v/lnQ|V«v)  ■  1.  (10) 

Given  Vcv,  let  G(v,S)  be  the  greatest  Integer  less  than  v/lnQ  If  $>0  and 
G(v,$)  ■  «,1f  S  ■  0. 

Wo  can  now  write  down  the  conditional  distribution  of  N  given  V«v  as 

p(njR)  s  Pr(N*n|V"V,  N>0) 
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where  n  ■  1,  2,  ....  G(v,$).  This  expression  for  p(niR)  can  be  readily  eval¬ 
uated  at  minimal  cost  with  an  electronic  computer. 

3.3  Inferential  Procedures 

A  conservative  l-o  confidence  Interval  for  ft  can  be  constructed  by 

G(v,$)  n 

obtaining  values  R,  arid  R«  which  satisfy  J  p(kiR.)  ■  o.  and  l  p(k*,R9)  ■  cu 

c  k*n  k*l  c  K 

where  a1  +  a2  *  a.  The  corresponding  confidence  bounds  fcr'R"1  (the  MTBF  et 

test  time  T+S)  are  Rg-1  and  R^”\.  Because  N  Is  a  discrete  random  variable* 

construction  of  exact  confidence  intervals  would  require  randomization.  A 
UMPU  test  of  H0:RsRq  versus  H^:R>RQ  at  significance  level  a  calls  for  rejection 

G(v.S)  ''  V  .  ■  '■■■,  . '  .. 

of  H0  If  l  p(kiRn)sa.  Other  UMPU  hypothesis  tests  can  be  constructed  In 

k»n  ■  jlfi-  " 

a  similar  manner.  If  Rq"1  Is  the  MTBF  goal  for  the  end  of  the  test  phase 
(time  T+S),  then  the  risk  of  not  achieving  this  goal  may  be  evaluated  as 

.I.PCk;  R0). 
k«l  u 

4.  EXAMPLE 


Suppose  that  a  reliability  growth  test  phase  has  been  In  progress  for 
T-20Q  hours  and  Is  scheduled  to  continue  for  another  S-200  hours.  From  the 
test  data  up  to  time  200,  we  wish  to  obtain  an  80  percent  confidence  Interval 
for  the  MTBF  at  time  T+S  ■  400.  The  following  failure  times  t^  were  recorded 

(n-21):  2.2,  3.3,  4.S,  5.3,  5.8,  20.3,  27.4,  34.1,  55.2,  58.4,  61.4,  62.2, 
78.3,  78.4,  91.9,  97.7,  112.4,  116.9,  142.4,  176.8,  181.5. 

Equations  (3)  and  (4)  yield  i  ■  .591  and  ft-1  *  21.4,  and  It  Is  also  of 
Interest  to  observe  that  v/lnQ  ■  72.3.  Thus  G(v,S)  ■  72,  so  that  the  *et  of 
positive  Integers  less  than  or  equal  to  72  Is  a  support  of  the  conditional 
distribution  of  N  given  V*v. 

With  Equation  (11)  we  obtain  by  Iteration  the  values  Rg"1  *  12.7  and 

-1  l2  V 

R-,  ■  38.6  such  that  l  p(k',R,)  ■  .,10  and  I  p(kiRJ  »  .10.  The  Interval 

1  k-21  '  k-1  z 

(12.7,  38.6)  Is  therefore  an  80  percent  confidence  Interval  for  the  MTBF  at 
time  400. 

By  successively  taking  S*0  and  5*100,  we  cap  obtain  In  a  similar  manner 
80  percent  confidence  Intervals  (10.7,  26.0)  and  (11.9,  32.7)  for  the  MTBF  at 
times  200  and  300,  respectively.  All  three  confidence  Intervals  are  shown  In 
Figure  2  for  comparison  purposes. 
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FIGURE  2.  Eighty  Percent  Confidence  Intervals  for  Current 
arid  Future  MTBF  Parameters. 


SK#iUrther  tfiat  ?"  goa1  of  15-°  h«*  b««"  *«t  as  a  milestone  for 

JJ?  ^  cZry  rSlJabi!1ty,growih  t#,t  Phaaa  <T+S  -  *00).  Based  on 
the  data  up  to  time  T  -  200,  the  risk  of  not  achieving  this  goal  Is 

J^PO^IB  1 )  ■  .20.  In  view  of  such  a  result,  the  program  manager  should  feel 

optimistic  about  this  aspect  of  the  development  program,  but  will  probably  wanl 
enhancemen^efforr1  Wh  Ch  w1flht  adv,fMly  affiqt  th#  ov«rall  reliability* 
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SPECIFYING  A  DETECTABLE  3- FACTOR 
INTERACTION  WITH  THE  NON-CENTRAL  F 

Walter  D.  Foster 
Jack  L.  WTay 

AFIP,  Washington  D.C.  20306 

ABSTRACT.  Just  as  the  "pumpkin  papers"  typewriter  was  identi¬ 
fied  ,ril™nrplausible  that  the  print-out  can  identify  a  small  (home) 
computer.  A  typical  print  head  consists  of  7  (or  9)  rods  or  pins 
in  a  vertical  column  which  move  in  time  to  7  successive  positions' 
in  the  character  space.  By  high  precision  photography  and  magni¬ 
fication,  density  readings  offer  an  approach  towards  identification 
of  a  specific  printer Ifrom  its  dot  matrix.  Because  ink  depletion 
on  e  ribbon  or  a  new  ribbon  on  the  same  printer  will  cause  changes 
in  density,  the  variable  of  analysis  was  taken  to  be  relative  densi¬ 
ty,  the  density  of  a  set  of  pins  relative  to  a  specific  pin.  Analyr 
sis  of  repeated  characters  per  printer  over  a  set  of  9  printers 
gave  rise  td  an  analysis  of  variance  in  which  a  3-factor  interaction 
term,  if  large  relative  to  the  within  term  (£or  repeated  deter¬ 
minations),  was  deemed  to  be  capable  of  excluding  printers.  A 
candidate  printer  is  identified  as  the  putative  originator  when 
the  3- factor  interaction  is  equal  to  the  within  mean  square.  Since 
identification  is  based  upon  accepting  the  null  hypothesis,  the 
non- central  F  distribution  was  used  to  set  a  value  for  the  3- factor 
interaction  under  the .concept  that  if  such  a  departure  existed,  the 
analysis  would  have  1  -  •  power  to  detect  it. 

1.  INTRODUCTION.  Tracing  ransom  ndtes  to  a  specific  typewriter 
primarily  has  depended  upon  finding  a  type  defect  in  the  document 
and  tracing  it  to  a  particular  machine.  This  tactic  is  successful 
when  the  defect  is  unusual  or  unique,  to  that  machine.  The  same 
technique  has  been  applied  to  print-outs  from  small  (home)  com¬ 
puters.  This  paper  is  concerned  with  a  statistical  model  for  the 
identification  of  a  computer  printer  that  is  operating  properly 

by  means  of  the  density  pattern  from  pin  to  pin  in  the  dot-matrix 
print-head  of  the  printer.  A  typical  print-head  consists  of  7  pins 
or  rods  in  a  vertical  column  which  move  horiiontally  in  a  character 
space  to  create  a  given  character.  Because  of  curvature  of-  the 
platen,  the  dots  in  the  print  matrix  tend  to  have  difterent  densi¬ 
ties.  A  well-worn  ribbon  leaves  different  densities  than  a  new 
ribbon.  Hence,'  density  of  itself  is  not  a  satisfactory  measure 
for  identification.  However,  the  difference  in  density  relative 
to  a  specified  pin  will  show  patterns  characteristic  of  a  print-head. 

2.  METHODS.  Measurements  of  density  were  obtained  by  making 

. high-precision,  photographic  enlargements  of  characters  on  a  print¬ 
out.  A  precision  densitometer  with  an  absolute  reference  cali¬ 
bration  was  used  to  measure  the  densities  of  pin  marks  on  the  print¬ 
out.  Vertical  characters  were  represented  by  1,  B,  D,  B  and  slant 
characters  by  7,  X,  Z,  /.  Determinations  of  relative  density  were 
made  on  each  of  five  pins  in  each  character  for  nine  printers.  The 
characters  were  printed  five  times  by  each  printer  to  provide 
replication. 
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3.  STATISTICAL  MODEL.  Main  factors  wart  defined  as  Exemplars 
(or  print-outs  from  printers),  Pins,  Verticality,  and  Characters 
in  vertical ity.  Characters  were  nested  in  verticality;  all  factors 
were  considered  as  fixed  effects  except  characters.  This  balanced 
model’ is  written  as: 

Y  ■  u+Re+Pp+Vy+Cgy+BPjp+BVgy+PVpy+EPV^py+EC^gy+PCpgytlPCgpgy+e^pgyj., 

Exemplars  as  a  main  effect  had  no  relevant  interpretation  because 
the  variable  of  analysis,  relative  density,  included  an  arbitrary 
reference  pin.  Verticality  was  defined  as  a  two- level • factor  com¬ 
prised  of  the  average  of  the  vertical  characters  and  the  average 
of  the  slant  characters.  The  pattern  of  Interest  was  the  Pin  x 
Verticality  Interaction  and  how  it  varied  from  one  exemplar  to 
another,  Figure  1.  Thus,  the  3 -factor  interaction,  Exemplar  x 
Pin  x  Verticality  was  taleen  as  the  criterion  for  Identifying  two 

?rint-outs  as  coming  from  the  same  print-head  or  conversely  deny- 
ng  that  they  had  a  common  source. 

i 

Denying  that  two  print-outs  could  have  come  from  the  same 
print-head  was  achieved  by  finding  a  statistically  significant  HPV 
interaction.  Thus  the  probability  of  making  a  Type  I  error  was 
fixed  at  *  . 

Affirming  that  two  print- outs  could  have  come  from  the  same 
print-head  was  equivalent  to  accepting  the  null  hypothesis.  In 
order  to  control  the  rate  of  Type  IX  errors,  a  ,  it  was  necessary 
to  specify  a  value  for  the  BPV  interaction,  that  would  be  detected 
if  it  existed  1  -  i proportion  of  the  time.  The  non-eentral  F 
parameter,  k 

2  ?  Cui  -  u)  /k 

0  »  -  .  *  ,  where  k  ■  #  means  and 

ffz/n  n  ■  f  values  per  mean) 

is  customarily  used  to  specify  a  set  of  means  under  the  alternate 
hypothesis.  Because  0Z  has  the  F  structure- -the  numerator  is  a 
mean  square  of  the  means  under  the  alternate  hypothesis  and* the 
denominator  is  the  variance  of  a  mean- -it  is  clear  that  0*  can 

be  used  to  specify  an  interaction  term  as  well. 

» 

It  is  not  immediately  clear  what  value  of  k  or  degrees  of 
freedom,  should  be  used  for  0  in  the  case  of  interaction.  We 
begin  our  argument  by  examining  a  2- factor,  2  x  C  interaction. 

Figure  2b.  Under  the  null  hypothesis,  the  F-statistic  tests 
whether  the  difference  between  two  rows  is  the  same  from  column  to 
column.  The  alternative  hypothesis  would  specify  a  pattern  of 
differences.  Therefore  it  is  argued  that  the  degrees  of  freedom 
for  the  2  x  C  Interaction  would  be  C,  one  for  each  column. 

The  case,  for  a  3-factor  interaction  composed  of  two  categories, 
two  rows/ category,  and  C  columns  follows  the  same  argument,  under 
the  null  hypothesis,  the  F-statistic  tests  whether  the  pattern  of 
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row  differences  across  columns  is  the  same  from  one  category  to 
Jh"  other,  Piguro  2c.  The  alternative  hypothesis  would  specify 
how  the  pattern  of  row  differences  across  columns  would  vary  from 

SSSts;5??:ryit  a  syras  awys  srssg  s$ 

expression*for'nf ‘  in  {h.  .JUi.l’cis.  of  a  2  *  2  *  C  interaction 
should  be  C. 

conven tionai* eoneep t*o f  samp* a° s  i se  tiinu,^  th?numb er'of^cons tr aints 

‘g.  r 

tlvll  constraints  on  the  mean,  so  that  k  -  C  •  0  ■  C.  In  Jlgura 
^b»w:f.?o.pip:t=0fi.d,s.r*thS-.u:sri:’tivc.  h4ypfotS..i.. 

SF.W'!  K“  WM.'U.fltt.S 

rsi  fc,  !■  A;*  a;u. 

A^iatesory.  The  number  of  constraint*  on  the  d*j  is  C, 
to  the*argument  for  2b.  Likewise,  the  number  oT  eon* triaintsi  on  the 
j  i  €  dif ftrtncoi  •  ■  din  %  rtc^uirt  th§t  tith^r  th# 

Jlluil  totals  for  one  categbr^3 or  thl  column  totals  over  both 
catSSSrlSi  bl  fixed^.o* that  the  number  of  constraint.  1.  C. 
Therefore,  k  ■  4C  -  C  •  C  C  ■  C. 

To  complete  the  statistical  model  using  5h#vn?neC/5o£atheFS 

MUn.&y“  ‘a  «li.Vf  ■( 

»  «yb.  .  Sboul!  be  set  rint- 

hi  d’ior  tSo4#ivSu«  Jith  .“significant!/  BPV  inrer.ctlon  and 
affirm” the  asm.  print-head  if  the  BPV  interaction  is  less  than 
2.72  times  the  Within  mean  square. 

n.ri.5A.SS^^^t??1of4vihr1lIn4:Cf«°?hriinr."mc"r.C«' 

\  ?  b.*#  arp.i.a.ars.wrisii  &..i. 

i«  itw^Jlth’p  i  ?Sl‘  Of  course,* th.  ek.mpl.r.  are  known  to  be 
different.  So  far  so  good. 

4n  interested  computernik  friend  offered  to  test  our  system. 

.f  p  v2:fl:*j!i!ttsnc,t1isl?lfe““.' -uni*  r<  sat 

a  *  ,03  for  F(4,  S20). 
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The  print-outs  were  photographed,  enlarged,  and  read  with  the  same 
densitometer  using  five  repeated  characters  as  before.  The  analysis 
of  variance  is  given  in  Table  1  under  the  heading  HWL345.  The 
pattern  for  the  EPV  interaction  is  shown  in  Figure  3  for  the  set 
marked  H3,  H4,  and  H5.  These  patterns  are  seen  to  be  remarkable 
similar  but  with  some  failure  to  be  exactly  the  same.  The  analysis 
of  variance  gave  an  F- ratio  of  1.82/5.67  for  the  BPV/Within  ratio, 
strongly  suggesting  that  the  three  print-outs  were  far  more  similar 
than  expected  according  to  experimental  technique.  The  decision 
rule  required  us  to  say  that  the  three  print-outs  all  .came  from  the 
same  print-head.  37  If  statisticians  on  occasion  refer  to  computer- 
niks  In  less  than  flattering  terms,  this  computernik  had  the  last 
word,  for  after  he  had  stopped  laughing  (some  300  x  10*  nanoseconds 
later  according  to  his  computer  clock),  he  revealed  that  the  three 
test  print-outs  indeed  came  from  three  separate  machines  but  of  the 
same  model.  Of  course  we  can't  tell  you  whose  mfg  these  machines 
were,  but  our  code  for  it  is  HWL.  At  any  rate,  we  had  flunked  this 
test. 

We  then  designed  our  own  test,  taking  two  successive  print- outs 
from  the  same  machine.  The  patterns  are  shown  in  Figure  2,  7D  and 
7B,  with  the  corresponding  analysis  of  variance  given  in  Table  1 
with  the  same  column  heading.  Again,  the  patterns  can  be  seen  to 
be  very  similar,  in  fact  too  similar  according  to  expectation  from 
the  Within  mean  square.  But  this  time  when  we  concluded  that  the 
two  print-outs  were  from  the  same  machine,  we  were' right. 

i 

Two  other  comparisons  are  shown  Tifcli'  l.;"  Exemplars  2  and  4 
were  chosen  at  random  for  comparison  and  found  to  have  an  P- ratio 
of  9.00/5.62  for  the  BPV/W  test.  It  comes  as  something  of  a  shock 
when  making  the  visual  comparison  of  exemplars  2  and  4  in  Figure  1 
to  realize  that  the  EPV  mean  square  merely  reflects  whether  the 
pattern  of  differences  in  exemplar  >2  is  equal  to  that  of  #4. 
Score-wise  for  this  comparison,  ouch, 

The  last  comparison  concerns  7AC,  whose  patterns  are  shown 
in  Figure  3.  The  EPV/W  ratio  was  found  to  be  16.83/9.8  ,  a  result 
that  affirms  that  the  print-heads  were  the  same.  They  were. 

Two  problems  have  surfaced.  First  is  the  failure  of  the 
within  meansquare  with  its  very  large  number  of  df  per  analysis 
to  remnin  reasonably  consistent.  Secondly,  it  may  well  be  that 
five  repeat  characters  is  not  enough  to  meet  the  stringency  this 
procedure  may  need. 


1/  Technically,  we  would  be  required  by  the  decision  rule 
to  males  three,  pair-wise  comparisons  of  the  exemplars  and  then 
to  live  with,  the  problem  of  dependent  comparisons. 


tablb  i.  ANALYSES  OF  VARIANCE  ON  DENSITY 


SUBSET  COMPARISONS 


7D 

vs  7B 

HWL 

-345 

7 -AC 

ALL 

NINE 

Sourco ’ 

df 

MS 

df 

MS 

df 

MS 

df 

MS 

lixomplar 

1 

.68 

2 

64.00 

1 

301.86 

8 

708.36 

Pins 

4 

174.08 

4 

47.25 

4 

516.26 

4 

100.19 

Vorticality  1 

777.50 

1 

536.79 

1  2291.94 

1 

96.20 

Char  in  V 

6 

34.33 

6 

50.79 

6 

115.16 

6 

34.52 

BP 

4 

2.92 

8 

2.69 

4 

27.05 

32 

86.28 

EV 

1 

199.08 

2 

49.01 

1 

44.70 

8 

86.86 

EC 

6 

47.52 

12 

57.63 

6 

126.69 

48 

39.95 

PV 

4 

26.65 

4 

3.88 

4 

39.95 

4 

17.99 

PC 

24 

3.93 

24 

5.50 

24 

5.47 

24 

6.57 

ItPV 

4 

1.60 

8 

1.82 

4 

16.83 

32 

6,77 

UPC 

24 

4.64 

48 

8.94 

24 

8.49 

192 

7.69 

With  in/ 5 

320 

7.00 

480 

5.67 

56 

9.80 

1440 

4.30 

2  vs  4 

Exomplar  1  3663.92 

Pins  4  178.31 

Varticality  1  161.88 

Char  in  V  6  19.01 

BP  4  177.14 

EV  1  254.18 

EC  6  33.09 

PV  4  6.78 

PC  24  6.99 

BPV.  4  9.00 

EPC  24  9.97 

Within/S  320  5.62 
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tha  authors  «id  ara  not  to  bo  ocoatruad  aa  official  or  aa  raflactlng  tha 
views  of  tha  Dapartsant  of  tha  Awy  or  tha  Dapartnant  of  Dafanaa. 
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SHOULD  CRITERIA  FOR  FIELD  TESTS 
BE  FORMULATED  AS  STATISTICAL  HYPOTHESES? 

Carl  T.  Russell 

US  Army  Cold  Regions  Tost  Canter 
Fort  Greely,  Alaska 

ABSTRACT.  This  paper  presents  for  discussion  an  example  typical  of 
many  faced  by  a  statistician  Involved  In  the  planning  and  analysis  of  Army 
field  tests.  The  example  raises  philosophical  and  procedural  questions 
concerning  the  degree  to  which  statistical  formalism  (especially  that  of 
hypothesis  testing)  should  be  applied  to  field  test  planning  and  analysis  In 
cases  where  numerical  criteria  are  given.  The  author  believes  that  although 
such  criteria  can  serve  as  useful  planning  guides.  It  Is  usually  foolish  to 
behave  as  If  such  criteria  can  reduce  test  objectives  to  tests  of  statis¬ 
tical  hypotheses.  This  belief  has  led  the  author  to  be  quite  casual  In  much 
of  his  statistical  planning  and  analysis,  as  the  example  shows.  The  author 
solicits  both  a  critique  of  his  approach  and  suggested  Improvements. 

I.  INTRODUCTION  AND  STATEMENT  OF  PROBLEM.  Winter  temperatures  below 
freezing  are  common  in  highly  industrialized  regions  of  Europe,  Russia,  Red 
China,  and  Korea.  Since  these  are  all  regions  where  the  U.S.  Army  could 
fight,  the  teed  for  a  test  canter  for  examining  performance  of  Army  per¬ 
sonnel  and  materiel  In  the  cold  Is  obvious.  The  U.S.  Arty  Cold  Regions  Test 
Center  (CTTC)  Is  located  at  Fort  Greely,  Alaska  where  temperatures  are  below 
32°F  more  than  80  percent  of  the  time  during  the  winter  months,  averages  49 
days  per  year  below  -25°F,  averages  32  days  per  year  below  -  4lrF,  and  aver¬ 
ages  an  annual  low  of  -59°F.  Although  colder  areas  exist  other  than  Fort 
Greely,  no  other  accessible  area  In  the  United  States  Is  available  to  the 
U.S.  Army  for  cold  regions  testing  of  military  systems  In  the  cold. 

One  system  recently  tested  at  CRTC  was  a  pyrotechnic  warning  cartridge 
(aerial  flare)  consisting  of  a  pyrotechnic  whistle  and  three  pyrotechnic 
stars.  Previous  experience  with  similar  pyrotechnics  had  shown  that 
storage,  transportation,  and  firing  at  subzero  temperatures  could  result  In 
severe  performance  degradation.  Two  hundred  cartridges  from  a  total  of 
approximately  2,500  prototype  cartrldes  manufactured  were  provided  for 
testing  at  CRTC.  Late  In  test  planning  It  was  determined  that  each  cart* 
ridge  would  be  enclosed  In  a  hermetically  sealed  can  and  that  these  cans 
would  be  packaged  In  cardboard  cartons  (unit  packs)  of  sight.  The  cart¬ 
ridges  arrived  at  CRTC  In  seven  wooden  boxes;  six  boxes  contained  four  unit 
packs  each  and  the  seventh  box  contained  one  unit  pack.  The  25  unit  packs 
were  labeled  "A"  through  “Y"  where  unit  packs  A  through  ti  came  from  the 
first  box,  E  through  H  came  from  the  second  box,  and  so  on.  To  avoid  con¬ 
fusion  during  test  execution  (outside,  at  subzero  temperatures)  virtually 
all  pretreatment  and  firing  was  planned  In  blocks  of  eight  cartridges,  and 
blocks  were  confounded  with  unit  packs. 

Among  the  criteria  for  test  were  numerical  criteria  for  physical  char¬ 
acteristics  (cartridge  size  and  weight)  and  numerical  criteria  for  perfor¬ 
mance  characterlsltlcs  (audible  and  visual  signal  timings  and  height  of 
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burst).  CRTC's  draft  Detailed  Teet  Plan  (DTP)  treated  both  types  of  cri¬ 
teria  rather  casually,  translating  them  Into  decision  rules  (which  could  be 
Interpreted  as  critical  regions  for  appropriate  hypothesis  tests)  rather 
than  formulating  them  as  hypotheses  for  test.  Higher  level  review  of  the 
DTP  produced  recommendations  for  a  more  formal  approach  which  were  only 
cosmetically  Incorporated  by  CRTC. 

In  the  author's  view,  data  generated  from  the  test  Indicated  that  a 
very  Informal  approach  to  analysis  was  appropriate.  In  order  to  address  the 
formal  Issues  raised  at  higher  level,  however,  a  moderately  thorough  formal 
analysis  was  conducted  and  partially  presented  In  the  test  report.  But  even 
the  limited  formality  of  the  analysis  presented  In  the  test  report  seems  (to 
the  author)  to  detract  from  the  simple  test  results  by  adding  unnecessary 
statistical  pedantry.  Simple  summary  descriptions  and  graphical  displays 
relating  results  to  criteria  would  have  sufficed. 

What  follows  Is  a  two-part  example  In  which  first  the  criteria  for 
physical  characteristics  than  the  criteria  for  performance  characteristics 
are  examined  In  terms  of  the  plans,  comments,  and  analysis  which  they  gen¬ 
erated.  The  example  Is  taken  almost  verbatim  from  test  documentation,  and 
It  raises  philosophical  and  procedural  questions  concerning  the  degree  to 
which  statistical  formalism  should  be  applied  to  field  test  planning  and 
analysis  In  cases  where  numerical  criteria  are  given.  In  a  statistically 
simple  world,  criteria  would  accurately  reflect  all  essential  system  char¬ 
acteristics,  planning  would  direct  testing  at  those  criteria,  and  the  only 
Important  test  results  would  be  whether  or  not  the  system  met  the  criteria 
and  at  what  level  of  statistical  significance.  The  author  does  not  deal 
with  such  a  world,  and  to  act  as  If  he  did  would  not  only  appear  very 
foolish  to  the  nonstatisticians  with  whom  he  works,  but  also  adversely  In¬ 
fluence  both  test  results  and  their  presentation.  However,  the  casual 
approach  to  criteria  documented  here  has  not  proven  to  be  acceptable  through' 
out  the  Army  statistical  community,  and  the  author  solicits  both  a  critique 
of  his  approach  and  suggested  Improvements. 

II.  PHYSICAL  CHARACTERISTICS  OF  CARTRIDGES. 


Criterion 


The  cartridge  shall  not  exceed  the  following  size  and  weight  limi¬ 
tations:  length  -  30.5  centimeters  (12  In);  diameter  -  40  millimeters  (1.58 
In);  weight  -  681  grams  (1.5  lbs). 

Planned  Test  Design/Procedure 

One  unit  pack  was  to  be  selected  at  random  and  all  eight  cartridges 
removed  from  their  sealed  cans,  weighed  and  measured.  These  cartridges  were 
to  be  used  In  initial  safety  firings. 
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Planned  Analysis 

Physical  dimensions  and  weights  will  be  presented  In  tabular  form.  The 
criterion  concerning  physical  characteristics  will  be  considered  met  If  the 
mean  Is  less  than  the  required  limits. 

Comments  from  Higher  Level  Review 

Comparing  the  mean  to  the  requl rement  Is  not  an  adequate  analysis.  The 
requirement  states  values  which  the  Item  shall  not  exceed.  The  mean  com¬ 
parison  would  allow  up  to  50  percent  of  the  Items  to  exceed  the  requirement. 
The  plan  should  Include  either  of  the  following: 

(a)  A  one-sided  test  of  hypothesis  at  some  crrlsk  level. 

(b)  Computation  of  a  one-sided  90  or  95  percent  tolerance  limit  at 
some  1-a  confidence  level  and  comparison  of  the  limit  to  the  required  value. 

Response  to  Higher  Level  Review 

Add  to  planned  analysis  the  sentence:  “Confidence  levels  will  be  given 
where  appropriate." 

Actual  Test  Oeslqn/Procedure 

As  planned. 

Results  and  Analysis 


TABLE  1.— Cartrldaa  Weights  and  Dimensions 


Cartridge 

Number 

Cap1" 

Diameter  (mml 

Cartridge 
Diameter  (mml 

1 

42.69 

39.90 

25.73 

492.0 

2 

42.78 

40.14* 

25.72 

494.5 

3 

41.92 

39.98 

25.73 

492.5 

4 

42.04 

39.96 

25.70 

491.0 

5 

41.40 

39.90 

25.75 

493.0 

6 

41  92 

40.06* 

25.32 

492.5 

7 

42.06 

40.04* 

25.73 

492,0 

8 

41.78 

40.02* 

25.71 

490.0 

Mean 

42.07 

40.00 

25.67 

492.2 

Std  Deviation 

0.46 

0.08 

0.14 

1.3 

"Exceeds  crlterl 

oh. 

(a)  The  criterion  concerning  physical  characteristics  was  considered 
met  since  all  lengths  and  weights  ware  well  below  the  criterion  values. 
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(b)  Although  the  criterion  did  not  specify  whether  cep  diameter  or 
cartridge  diameter  was  to  be  less  than  40  millimeters  (1.5748  In),  It  was 
assumed  that  cartridge  diameter  was  most  relevant.  The  mean  cartridge 
diameter  was  exactly  40.00  millimeters.  Basing  a  confidence  Interval  on  the 
t-statlstlc,  It  can  be  stated  with  99  percent  confidence  that  the  mean 
cartridge  diameter  Is  between  39.89  millimeters  and  40.11  millimeters 
(1.570  In  and  1.579  In). 

III.  PERFORMANCE  CHARACTERISTICS  OP  CARTRIDGES. 

Criteria 


(a)  Audible  and  visual  signals  of  maximum  Intensity  shall  be  produced 
within  7  seconds  after  the  cartridge  has  been  triggered  to  fire. 

(b)  When  launched  vertically  upward,  the  cartridge  shall  have  the 
capability  of  producing  audible  and  visual  signals  at  a  minimum  altitude 
above  the  launch  site  of  50C  feet  (152.4  meters). 

(c)  The  duration  of  the  audible  signal  and  the  duration  of  the  visual 
signal  shall  each  be  at  least  5  seconds.  Signal  durations  greater  than  7 
seconds  will  not  serve  a  useful  purpose. 

Planned  Test  Desl on/Procedure 

Times  were  to  be  obtained  by  a  ground  observer  using  a  stopwatch,  and 
It  was  hoped  that  burst  locations  could  bt  obtalnad  using  a  video  scoring 
system  developed  at  CRTC.  Experimental  desldn  complications  arose  since  In 
addition  to  examining  the  performance  criteria  glren  above  at  varying  tem¬ 
peratures  and  with  or  without  transportation  of  rounds  prior  to  firing,  the 
visibility  and  audibility  of  signals  at  various  distances  from  the  launch 
site  under  various  light  and  weather  conditions  were  of  concern,  as  was  the 
ability  of  various  personnel  to  fire  the  cartridge  wearing  various  cold 
weather  gear.  The  hope  was  to  rotate  seven  observers  through  six  observer 
positions  and  ona  firing  position  and  to  fire  cartridges  both  day  end  night, 
both  clear  end  snowing,  end  both  calm  and  windy.  In  addition,  safety  consi¬ 
derations  necessitated  plans  to  fire  at  least  20  cartridges  under  «n y  tem¬ 
perature  condition  prior  to  hand  firing  at  that  temperature.  With  200  cart¬ 
ridges  available  for  firing,  —  essentially  In  blocks  of  eight  --  no  stan¬ 
dard  balanced  design  approach  seemed  feasible.  The  design  approach  taken 
was  a  traditional  approach  varying  one  selected  factor  at  a  time  as  sum¬ 
marized  In  table  2. 
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TABLE  2.— Planned  Flrlnt 


Number1 


Trial 

of 

Rounds 

Temper¬ 

ature 

Day¬ 

light 

Snow 

Wind 

i*t£l 

Flrer 

Transpor¬ 

tation 

Human* 

Factors 

1 

4 

1  >0°F 

Y 

■  H 

<10 

R* 

N 

Y 

2 

4 

>Q*F 

N 

N 

<10 

R* 

N 

Y 

"  3  >■ 

.  4 

8 

01* 

Y  , 

N 

<10 

Ra 

Y 

Y 

16 

01* 

Y 

N 

<10 

R* 

N 

Y 

5 

8 

C1» 

N 

N 

<10 

Ra 

Y 

Y 

6 

8 

Cl* 

N 

N 

<10 

R* 

N 

Y 

7 

,  -8 

Cl* 

Y 

Y 

<10 

R 1 

N 

Y 

8 

'  6 

Cl* 

N 

Y 

<10 

R 

N 

Y 

-.9.  ■ 

8 

Cl® 

Y 

N 

10-20 

R 

N 

Y 

10 

8 

Cl* 

Y 

N 

<10 

A4 

N 

N 

11  , 

8 

Cl* 

Y 

N 

<10 

B4 

N 

N 

12  , 

8 

Cl* 

Y 

N 

<10 

C4 

N 

N 

13 

8 

Cl* 

Y 

N 

<10 

D4 

N 

N 

14 

8 

Cl* 

,  Y 

N 

N 

<10 

E4 

N 

N 

15 

8 

Cl® 

Y 

<10 

F4 

N 

N 

16 

'  ""'8 

Cl® 

Y 

N 

<10 

fi4 

N 

N 

17 

8 

16 

C2* 

Y” 

N 

<10 

R« 

Y 

Y 

18 

C2*  - 

Y 

N 

<10 

Ri! 

N 

Y 

19 

8 

C2* 

N 

N 

<10 

R* 

Y 

Y 

20 

8 

C2* 

N 

N 

<10 

Ra 

N 

V 

21 

16 

C3* 

Y 

N 

<10 

R2 

N 

Y 

22 

8 

C3* 

N 

N 

<10 

Ra 

N 

Y 

1  Eight  cartridges  aval labl*  it  replacement  for  any  trial. 

4  Safety  will  ba  assessed  for  aach  firing;  however  remote  firings  ara  speci¬ 
fically  to  confirm  tha  safaty  of  firing  at  a  naw  low  tanperatura. 

3  Human  factors  evaluatad  as  a  part  of  aach  firing.  Timing  for  CB  protec¬ 
tive  ensemble  will  ba  recorded  as  wall. 

4  Two  rounds  fired  with  chemical  protective  handgaar,  two  rounds  with  trig¬ 
ger  finger  mittens,  and  four  rounds  fired  with  arctic  mittens. 

5  Climatic  design  types  as  defined  In  AR  70-36: 

Cl  (basic  cold)  -  -5°F  to  -25#F 
C2  (cold)  -  -35*F  to  -50°F 
C3  (severe  cold)-  below  -60°F 


(a)  Criterion  (a)  will  be  met  If  95  percent  of  the  functioning  cart¬ 
ridges  provide  audible  and  visual  signals  within  7  seconds  after  the  cart¬ 
ridge  has  been  triggered  to  fire. 
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(b)  Criterion  (b)  Will  be  met  If  95  percent  of  the  functioning  cart¬ 
ridges  produce  audible  and  visual  signals  at  a  minimum  altitude  of  500  feet 
above  the  launch  site. 

(c)  Criterion  (c)  will  be  met  if  95  percent  of  the  functioning  cart¬ 
ridges  produce  audible  and  visual  signals  of  a  5-second  duration  each. 

Comment  from  Higher  Level  Review 

The  analysis  should  be  more  than  "met  and  not  met"  decision  rules.  How 
will  the  data  be  Analyzed  and  presented?  The  analysis  should  address  such 
things  as  whistle  duration  vs.  temperature  and  differences  In  cartridge 
performance  after  being  transported. 

Response  to  Higher  Laval  Review 

Add  to  planned  analysis  the  paragraph:  "The  data  will  be  examined  for 
trends  and  exceptional  values,  and  substantive  findings  will  be  discussed." 

Actual  Test  Design/Procedure 

Exceptionally  warm  weather  after  the  test  Items  arrived  forced  testing 
to  be  done  quickly  when  cold  temperatures  were  available.  No  firing  was 
conducted  In  winds  exceeding  5  knots.  Observers  were  not  rotated,  but  25 
different  observers  were  used.  Only  sixteen  cartridges  were  fired  hand-held 
due  to  lateness  of  an  appropriate  safety  release,  and  burst  locations  were 
not  measured  for  the  hand-held  launches.  The  video  scoring  system  was 
unavailable,  so  burst  locations  were  determined  from  azimuth  and  elevation 
observations  taken  by  qualified  personnel  at  five  ground  observation  points. 
Different  personnel  measured  signal  timings  on  different  days.  The  actual 
design  executed  Is  summarized  In  table  3. 
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TABLE  3,~ActU3l  Firing  Matrix 


Trial 

Number 

of 

Rounds 

Ambient  Air 
Tempera¬ 
ture  (*F)» 

Day- 

llght 

Remarks 

1 

4 

26 

Y 

2 

4 

26 

N 

a 

3 

8 

-18 

Y 

10 

4 

16 

-25 

Y 

5 

8 

-12 

N 

9  10 

a 

6 

8 

-10 

N 

7 

6 

-10 

Y 

9 

8 

8 

-10 

N 

9 

8 

30 

Y 

4  9a 

»  » 

10 

8 

-23 

Y 

11 

8 

-22 

Y 

12 

8 

-22 

Y 

13 

8 

-20 

Y 

14 

8 

-20 

Y 

15 

8 

-16 

Y 

7 

16 

8 

-17 

Y 

17 

8 

-36 

Y 

11  12 

9 

18 

16 

-39 

Y 

u 

19 

8 

-36 

N 

20 

8 

-39 

N 

21 

16 

-22 

Y 

12  13 

12  13 

22 

8 

-22 

N 

23 

8 

-22 

Y 

S  •.  1 

2  3 


S«f«ty  was  ataasaad  for  each  firing;  however  remote  firings  were  spaclf le¬ 
al  ly  to  confirm  tha  safaty  of  firing  at  low  tamparaturas. 

Observer  human  factors  avaluatad  as  a  part  of  aach  firing. 

Four  rounds  wars  flrtd  hand-hsld  with  leather  work  gloves,  two  rounds 
wara  flrad  with  trlggar  flngar  mittens,  and  two  rounds  wara  flrad  with 
arctic  mlttans. 

Flrad,  hand-hald,  with  laathar  work  glovas. 

Obsarvars  unwarnad. 

Ona  round  was  "no  tast." 

Rounds  condltlonad  to  -40#F, 

Flrad  In  falling  snow. 

Rounds  transported  100  ml las  prior  to  firing. 

Rounds  transported  SO  ml  las  prior  to  firing. 

Sound  readings  obtained. 

Rounds  conditioned  to  -55°F  prior  to  firing. 


s 

« 

7 

8 

9 

to 

tt 

12 

13 
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Results  and  Analysis 

(a)  All  cartridges  exceeded  500  fast  (152.4  maters)  height  of  burst 
(HOB)  except  for  one  which  had  e  378  feet  (155  meters)  HOB.  All  functioned 
at  peak  trajectory  and  all  residue  extinguished  prior  to  Impact. 

(b)  Table  4  summarizes  cartridge  performance  for  the  time  to  audible 
and  visual  signal  Initiation  (maximum  signal  Intensity  was  obtained  almost 
simultaneously  with  signal  Initiation  and  could  not  be  separately  timed) , 
audible  and  visual  signal  duration,  HOB,  and  burst  deviation  from  vertical. 
A  round- by- round  listing  of  performance  data  appears  at  appendix  A. 

TABLE  4.— Performance  Data  Summary 


Time  to 

Audible 

Visual 

Deviation 

Signal 

Signal 

Signal 

from 

Initiation 

Duration 

Duration 

HOB 

Vertical 

Statistic 

(seconds) 

(seconds) 

(seconds) 

(feet) 

ld.eflra.es) 

Minimum 

3.9 

6.1 

6.0 

378 

0.9 

Maximum 

6.5 

13.0 

11.0 

803 

34.8* 

Median 

5.0 

7.2 

7.7 

672 

6.3 

Mean 

5.0 

7.4 

7.7 

667 

7.6 

No.  Observed  1951  . 

195* 

197s 

1834 

1834 

No.  Satisfactory  195* 

195* 

197® 

182’ 

133®,  * 

Percent, 

Satisfactory  100.0 

100.0 

100.0 

99.5 

72.7 

1  Excludes 

rounds  trial 

7,  Nos.  l  &  3;  trial  9,  nos. 

1  &  2;  and 

trial  is , 

No.  8. 

a  Excludes 

rounds  trial 

9,  No.  5;  trial 

11,  Nos.  1 

,  4,  8;  and  trial  15, 

No.  8. 

3  Excludes 

rounds  trial 

2,  No.  4;  trial 

9,  No.  1; 

and  trial  15,  No.  8. 

4  Excludes 

16  hand-held 

rounds  of  trials 

9  and  23; 

as  well  as 

"no  test" 

round  of  trial  15,  No.  8. 

8  Number  less  than  7  seconds. 

8  Number  greater  than  5  seconds. 

7  Number  greater  than  500  feet.  Low  round  trial  22,  No.  2  was  cold  condi¬ 
tioned. 

8  Number  less  than  10  degrees  from  vertical. 

g  All  but  round  trial  22,  No  2  were  less  then  19  degrees  from  vertical. 

(c)  Criteria  (a),  (b),  and  (c)  were  satisfied  since  the  cartridge 
performed  as  designed.  The  data  were  examined  for  trends,  but  no  substan¬ 
tial  trends  were  found.  A  statistical  discussion  appears  at  appendix  B. 


258 


APPENDIX  A 

Performance  Characteristics 


Tlwe  to 

Audible 

Visuil 

Haight 

Deviation 

Signal 

Initiation 

Signal 

Duration 

Signal 

Duration 

of 

Burst1 2 

fro* 

Vertical* 

Firing 

Trial 

Round 

Ltesl 

(Sac) 

(Deareas) 

Order 

1 

5 

4.0 

7.0 

7.6 

755.3 

2.0 

1 

1 

6 

4.0 

7.0 

7.0 

803,4 

2.7 

2 

1 

7 

4.0 

6.2 

8.0 

752.0 

2.4 

3 

1 

a 

4.5 

6.3 

7.8 

750.8 

4.0 

4 

2 

3 

S.O 

6.9 

7.0 

771.5 

3.0 

5 

2 

4 

5.0 

8.0 

NR 

746.5 

4.3 

6 

2 

5 

5.0 

7.5 

6.5 

752.3 

1.5 

7 

2 

6 

5.0 

8,0 

6.5 

750.8 

2.5 

8 

3 

1 

5.3 

7.4 

7.1 

688.6 

2.4 

81 

3 

2 

4.6 

7.1 

6.7 

683.8 

3.7 

82 

3 

3 

4.8 

7.0 

6.1 

682.1 

3.5 

83 

3 

4 

5.5 

7.3 

6.3 

669.2 

0.9 

84 

3 

5 

5.3 

7.3 

7.8 

682.1 

1.7 

85 

3 

6 

5.1 

7.3 

6.7 

683.3 

1.6 

86 

3 

7 

5.0 

7.6 

6.8 

6.6 

650.4 

3.3 

87 

3 

8 

5.4 

7.8 

669.5 

2.7 

88 

4 

1 

6.0 

6.1 

8.5 

675.7 

8.0 

25 

4 

2 

5.0 

6.6 

8.2 

682.7 

8.1 

26 

4 

3 

5.0 

6.4 

8.5 

793.6 

3.9 

27 

4 

4 

6.0 

7.8 

8.5 

673.2 

5.0 

28 

4 

5 

5.0 

7.0 

8.0 

783.4 

4.8 

29 

4 

6 

5.0 

6.9 

7.0 

716.7 

5.9 

30 

4 

7 

5.0 

7.3 

7.6 

692.7 

5.6 

31 

4 

8 

5.0 

6.6 

8.3 

621.0 

8.6 

32 

4 

9 

5.0 

7.2 

6.5 

691.9 

4.4 

33 

4 

10 

5.0 

7.1 

9.0 

646.8 

5.3 

34 

4 

11 

5.0 

7.0 

7.P 

664.1 

3.0 

35 

4 

12 

5.6 

7.4 

7.2 

680.1 

7.1 

36 

4 

13 

5.0 

7.9 

8.1 

684.9 

8.1 

37 

4 

14 

4.6 

7.7 

7.1 

679.5 

5.2 

38 

4 

18 

4.4 

7.4 

8.3 

674.0 

6.2 

39 

4 

16 

4.4 

7.6 

7.7 

684.3 

2.0 

40 

1  Mean  or  valuta  cETcuTvEicTTor  pair*  of  oEsirvirs  present  (weighted  by  tne 
number  of  observers  Involved).  Tha  naan  was  used  bacauta  valuaa  for  dlf- 
farant  pairs  wart  In  good  agramant. 

2  Hadlan  of  values  calculated  for  pairs  of  obsarvars  prasant.  Tha  wadi  an 
was  usad  bacausa  occasional  large  differences  between  pairs  occurred  In 
tha  data. 

NR  -  Not  Recorded 

A-l 
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Parforganca  Charactarl sties  (Continued 


Tima  to 

Audlbla 

Visual 

Haight 

Davlatlon 

Signal 

Initiation 

Signal 

Duration 

Signal 

Duration 

of 

Burst1 

from 

Vartleal1 

Firing 

Trial 

Round 

Ik £i 

Lksl 

iEUtt 

ikassssl 

Ordtr 

5 

1 

4.8 

7.0 

7.8 

710.4 

1.5 

105 

5 

2 

4.9 

7.0 

7.3 

654.8 

6.0 

106 

5 

3 

5.2 

7,1 

8,1 

650.8 

5.5 

107 

5 

4 

5.5 

6.9 

7.6 

699.9 

6.3 

108 

5 

5 

4.6 

8.3 

7.6 

717.0 

1,8 

109 

5 

6 

5.0 

7.7 

7.0 

722.4 

9.2 

110 

5 

7 

5.2 

7.8 

7.8 

686.0 

8,4 

111 

5 

8 

5.0 

6.6 

7.0 

717.3 

1.6 

112 

6 

1 

5.5 

6.6 

6.0 

662.1 

3.2 

9 

6 

2 

5.0 

7.2 

8.1 

687.3 

4.9 

10 

6 

3 

5,0 

6.6 

8.1 

656.4 

3.5 

11 

6 

4 

5.5 

7.0 

7.2 

676.0 

1.3 

12 

6 

5 

5.5 

7.5 

8,0 

716.6 

2.4 

13 

6 

6 

5.0 

7.0 

7.8 

737.9 

6.8 

14 

6 

7 

5.2 

7.0 

8.0 

715.7 

5.4 

15 

6 

8 

4.2 

7.5 

7,5 

672.9 

6.1 

16 

7 

1 

NR 

6,8 

8.3 

672.8 

6,0 

113 

7 

2 

5.8 

6.6 

7.6 

647.7 

7.9 

114 

7 

3 

NR 

6.8 

7.6 

686.5 

5.0 

115 

7 

4 

4.5 

7.3 

8.0 

663.0 

7.5 

116 

7 

8 

4.7 

7.0 

7.8 

546.9 

9.0 

117 

7 

6 

4.7 

7.8 

7.7 

691.2 

2.6. 

118 

7 

7 

4.5 

7,8 

7,6 

634.1 

5.4 

119 

7 

8 

5.3 

6,1 

7;8 

677.1 

1.9 

120 

8 

1 

5.0 

6.6 

8.5 

707.9 

5.9 

17 

8 

2 

5.0 

6.7 

7.8 

690.8 

2.7 

18 

8 

3 

5.2 

6.9 

7.2 

695.4 

2.8 

19 

8 

4 

5.0 

6.7 

8.0 

693.3 

5.0 

20 

8 

5 

5.2 

6.9 

7.8 

651.2 

6.2 

21 

8 

6 

5.8 

7,5 

7.2 

703.3 

3.5 

22 

8 

7 

5.0 

7.0 

8.0 

706.4 

3.7 

23 

8 

8 

5.0 

6.7 

7.2 

700.4 

5.3 

24 

numbtr  of  obsarvars  Involvad).  Tha  moan  was  usad  bacausa  valuas  for  dlf- 
farant  pairs  wars  In  good  agraamant, 

8  Hadlan  of  valuas  calculatad  for  pairs  of  obsarvars  prasant.  Tha  Radian 
was  usad  bacausa  occasional  larga  dlffaraneas  batwaan  pairs  occurrad  In 
tha  data. 

NR  -  Not  Racordad 


Parformanca  Charactarl sties  (Continued) 


Tina  to 

Audlbla 

Visual 

Haight 

Oavlatlon 

Signal 

Initiation 

Signal 

Duration 

Signal 

Duration 

of 

Burst1 

from 

Vortical1 

Firing 

Trial 

Round 

(Sac) 

ISscj 

if£lU 

Ordar 

9 

1 

NR 

7.0 

NR 

NR 

NR 

193 

9 

2 

NR 

6.5 

7.0 

NR 

NR 

194 

9 

3 

6.5 

7.0 

11.0 

Nr 

NR 

195 

9 

4 

6.9 

7.0 

7.0 

NR 

NR 

196 

9 

S 

6.0 

NR 

6.5 

NR 

NR 

197 

9 

6 

6.0 

7.2 

7.0 

NR 

NR 

198 

9 

7 

6.0 

6.8 

6.0 

NR 

NR 

199 

9 

8 

6.5 

6.8 

7.0 

NR 

NR 

200 

10 

1 

5.0 

7.1 

7.8 

627.2 

7.8 

41 

10 

2 

5.6 

7.1 

7.9 

717.8 

7.6 

42 

10 

3 

5.4 

6.9 

7.8 

671.8 

13.6 

43 

10 

4 

5.0 

6.4 

7.9 

712.9 

4.0 

44 

10 

S 

5.0 

7.3 

8.5 

686.8 

3.0 

45 

10 

6 

5.4 

7.0 

7.3 

682.8 

3.6 

46 

10 

7 

5.1 

7.2 

7.3 

672.7 

3.4 

47 

10 

B 

4.7 

7.7 

6.7 

689.8 

4.2 

48 

11 

1 

5.0 

NR 

8.1 

683.9 

4.7 

49 

11 

2 

4.5 

7.1 

7.3 

628.9 

8.1 

50 

11 

3 

5.5 

6.9 

7.2 

673.7 

9.4 

51 

11 

4 

5.5 

NR 

8.3 

659.8 

2.3 

52 

11 

5 

4.3 

7.6 

7.6 

630.2 

5.8 

53 

11 

6 

5.1 

7.1 

8.3 

574.0 

7.7 

54 

11 

7 

5.0 

7.1 

7.8 

668. 3 

6.5 

55 

11 

8 

5.1 

NR 

8.4 

662.5 

4.3 

56 

12 

1 

4.5 

6.7 

7.6 

629.3 

4.9 

57 

12 

2 

5.3 

6.9 

7.7 

696.4 

3.4 

58 

12 

3 

5.1 

6.7 

7.1 

686.5 

5.2 

59 

12 

4 

5.0 

7.2 

7.3 

694.0 

1.8 

60 

12 

5 

5.1 

7.3 

7.3 

649.4 

2.5 

61 

12 

6 

6.1 

6.6 

7.3 

662.3 

12.2 

62 

12 

7 

5.1 

7.7 

8.3 

656.8 

4.6 

63 

12 

8 

4.6 

6.5 

8.1 

677.1 

4.0 

64 

1  Maan  of  valuat  eiTcuTiCiH  for  pairs  of  obsarvars  prasani  (walghtad  by  tha 
numbar  of  obsarvars  Involved).  Tha  mean  was  usad  becausa  values  for  dif¬ 
ferent  pairs  were  In  good  agreement. 

1  Median  of  values  calculated  for  pair*  of  observers  present.  The  median 
wii  used  because  occasional  larga  dlffaraneas  between  pairs  occurrad  In 
tha  data. 

HR  -  Not  racordad 
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Performance  Charactarlatlce  (Continued 


Tima  to 

Audible 

Vlaual 

Haight 

Deviation 

Signal 

Initiation 

Signal 

Duration 

Signal 

Duration 

of 

Buret.1 

from 

Vertical1 

Firing 

Trial 

Round 

(Sac) 

(Sac) 

Qrdtr 

13 

1 

4.7 

7.3 

7.8 

649.1 

5.5 

65 

13 

2 

5.5 

7.0 

7.3 

662.9 

6.5 

66 

13 

3 

5.0 

8.1 

7.5 

662,0 

11.3 

67 

13 

4 

5.2 

6.8 

8.5 

594.2 

11.1 

68 

13 

5 

•'  4.5 

7.5 

7.9 

595.9 

8.8 

69 

13 

6 

4.5 

8.0 

6.9 

661.4 

4.7 

70 

13 

7 

4.7 

7.5 

7.5 

69!>.2 

5.8 

71 

13 

8 

5.0 

6.9 

8.1 

641.1 

2.1 

72 

14 

1 

5.6 

6.3 

7.1 

528.4 

9.4 

73 

14 

2 

4.2 

7.5 

7.3 

60S.0 

2.7 

74 

14 

3 

5.2 

7.7 

7.3 

623.7 

3.0 

75 

14 

4 

4.9 

7.9 

6.9 

670.9 

2.8 

76 

14 

5 

4.2 

7.3 

7.0 

605.7 

5.0 

77 

14 

6 

4.4 

7.3 

6.9 

654.0 

2.1 

78 

14 

7 

4.9 

8.0 

7.4 

635.9 

6.6 

79 

14 

8 

4.9 

7.7 

8.3 

670.1 

6.2 

80 

15 

1 

4.9 

8.0 

6.1 

655.7 

1.2 

89 

15 

2 

5.4 

7.3 

6.7 

688.7 

3.6 

90 

15 

3 

5.4 

8.1 

7.1 

704.0 

10.0 

91 

15 

4 

5.2 

7.7 

6.8 

622.3 

5.2 

92 

15 

5 

5.0 

7.3 

6.7 

703.0 

3.1 

93 

15 

6 

4.8 

7.0 

6.7 

653.3 

4.8 

94 

15 

7 

5.6 

7.8 

6.8 

608.4 

6.8 

95 

15 

8 

NR 

NR 

NR 

NR 

NR 

96 

16 

1 

4.6 

7.3 

6.9 

651.5 

1.8 

97 

16 

2 

5.6 

7.7 

7.9 

693.4 

2.7 

98 

16 

3 

4.8 

6.9 

7.0 

663.0 

6.6 

99 

16 

4 

4.9 

7.7 

6.9 

643.0 

4.9 

100 

16 

5 

6.4 

6.8 

6.9 

687.4 

2.3 

101 

16 

6 

4.5 

7.3 

6.9 

607.9 

5.4 

102 

16 

7 

4.9 

7.7 

6.3 

666.0 

4.7 

103 

16 

8 

5.3 

7.1 

6.1 

686.9 

6.5 

104 

1  Mean  of  valuee  calculated  for  pairs  of  obaervtre  praient  (weighted  by  tha 
number  of  obaervera  Involved).  Tha  mean  waa  uaad  bacauaa  valuta  for  dlf- 
farant  pair*  wart  In  good  agraamant. 

8  Madlan  of  valuaa  calculated  for  palra  of  obaervera  praaant.  Tha  madlan 
waa  used  bacauaa  occaalonal  larga  dlffarancaa  batwaan  palra  occurred  In 
tha  data. 

NR  -  Not  recorded 
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Performance  Character latica  (Continued! 

i? 

£ 

r 

i 

A 

Tima  to 

Audible 

Visual 

Haight 

Deviation 

1 

Signal 

Initiation 

Signal 

Signal 

of 

from 

I 

j 

Trial 

Round 

Duration  Duration 

Burat1 

Vertical2 

Firing 

3 

i 

'r  i 

ISaci 

ISlSl 

{Sec} 

lESttl 

LPftflmO 

Order 

J 

l  i 

17 

1 

4.9 

6.9 

7.0 

641.2 

13.1 

153 

Jj 

/•  1 

17 

2 

5.3 

7.5 

7.9 

666.6 

11.9 

154 

$1 

i)  1 

17 

3 

4.9 

8.1 

8.5 

668.5 

10.5 

155 

| 

*■; .  ( 

17 

4 

4.7 

7.7 

7.3 

638.9 

15.1 

156 

j 

17 

5 

4.6 

8.5 

7.3 

585.8 

15,3 

157 

•>:! 

i 

17 

6 

4.2 

7.8 

7.6 

587,3 

8.9 

158 

j.»  i 

17 

7 

3.9 

7.6 

8.1 

600.1 

8.9 

159 

■:) 

!/ 1‘  i 

17 

8 

5.4 

7.5 

7.8 

678.4 

10.8 

160 

R 

$ 

?  i 

18 

1 

5.0 

7.0 

7.5 

696.3 

13.8 

129 

’ll 

jT  5 

18 

2 

4.7 

7.0 

7.6 

601.3 

18.5 

130 

} 

*  j 

18 

3 

5.5 

7.0 

8.2 

598.7 

18.6 

131 

t,,  ! 

18 

4 

4.9 

7.0 

7.3 

699.0 

3.4 

132 

1 

i 

i 

18 

5 

4.9 

7.0 

8,2 

609.6 

10.1 

133 

18 

6 

5.1 

7.0 

8.1 

689.4 

10.1 

134 

■II 

fV  !■ 

18 

7 

4.8 

7.0 

7.2 

689.  C 

10.8 

135 

1 

18 

8 

5.7 

7.0 

7.7 

677.7 

8.6 

136 

4 

'll 

1 

18 

9 

4.6 

7.0 

7.9 

652.7 

13.9 

137 

f; .  ! 

18 

10 

5.1 

7.0 

7.7 

725.8 

11.0 

138 

I 

18 

11 

4.9 

7.0 

7.7 

644.9 

15.0 

139 

I 

18 

12 

4.9 

7.0 

7.8 

719.0 

9.9 

140 

1 

18 

13 

4.7 

7.0 

7.2 

647.6 

15.2 

141 

1 

i1" 

18 

14 

5.1 

7.0 

8.1 

661.4 

9.3 

142 

r 

18 

15 

5.5 

7.0 

10.6 

707.8 

9.3 

143 

$ 

' 

& 

18 

16 

5.3 

7.0 

7.8 

627.9 

2.6 

144 

1 

i 

19 

1 

4.9 

8.5 

8.3 

708.9 

13.4 

145 

1 

19 

2 

6.2 

7.2 

8.7 

682.2 

9.2 

146 

f 

19 

3 

4.8 

7.7 

8.9 

670.0 

11.0 

147 

1 

? 

19 

4 

5.3 

7.5 

8.3 

637.2 

10.3 

148 

1 

i !. 

19 

5 

5.2 

7.9 

7.7 

670.2 

8.0 

149 

1 

19 

6 

5.1 

6.9 

8.0 

649.6 

17.1 

150 

| 

1 

19 

7 

5.4 

7.2 

7.9 

684.5 

6.7 

151 

\ 

f 

19 

8 

4.6 

7.0 

7.9 

646.1 

14.0 

152 

J 

*  naan  ot  values  calculated  Tor  palra  or  o&aarvar*  present  (welghteaby  the 

lr 

numbar  of  observers  Involved).  Tha 

mean  waa 

uaad  because  valuaa 

for  dlf- 

ferent  palra 

ware  In  good  agreement. 

\ 

*  Madlan  of  values  calculated  for  palra  of  observers 

preaant.  Tha  madlan 

i  | 

waa 

uaad  bacauaa  occaalonal  large  dlffarancaa  batwaan  palra  occurred  In 

V  ! 

h  ? 

1 

i:  i 

\  ; 

t 

tha 

data. 

!i 

1  i 

I  s 
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Ptrfonwanct  Charactarl sties  (Continued) 


Tima  to 

Audible 

Visual 

Haight 

Davlatlon 

Signal 

Initiation 

Signal 

Duration 

Signal 

Duration 

of 

Burst1 

from 

Vertical* 

Firing 

Trial 

Round 

ISacl 

ISacl 

(Sac) 

IFaatl 

(Decrees) 

Ordar 

20 

1 

4.8 

8.0 

7.4 

722.2 

10.5 

121 

20 

2 

4.8 

msm 

7.0 

684.5 

14.8 

122 

20 

3 

5.0 

181 

7.1 

646.1 

16.7 

123 

20 

4 

4.9 

131- 

8.4 

707.5 

13.3 

124 

20 

5 

5.7 

8.0 

671.9 

12.3 

125 

20 

6 

5.0 

KSl 

8.0 

698.1 

11.7 

126 

20 

7 

4.6 

smi 

7.8 

753.2 

11.3 

127 

20 

8 

5.3 

7.0 

7.0 

699.8 

15.4 

128 

21 

1 

4.6 

8.1 

8.6 

638.1 

8.9 

177 

21 

2 

4.9 

7.9 

8.7 

634.7 

9.5 

178 

21 

3 

5.5 

7.3 

8.0 

686.1 

7.9 

179 

21 

4 

5.5 

7.9 

659.9 

8.8 

180 

21 

5 

4.5 

8.1 

7.9 

663.5 

12.8 

181 

21 

6 

4.9 

7.9 

7.7 

647.2 

15.4 

182 

21 

7 

5.0 

7.7 

8.5 

12.3 

183 

21 

8 

5.3 

MTpj 

8.0 

679.4 

9.4 

184 

21 

9 

4.8 

7.6 

8.7 

578.7 

16.4 

185 

21 

10 

4.0 

7.9 

7.5 

607.5 

15.2 

186 

21 

11 

4.9 

7.7 

628.9 

14.8 

187 

21 

12 

4.8 

7.3 

8.3 

599.9 

16.5 

188 

21 

13 

4.9 

7.7 

8.0 

545.5 

18.1 

189 

21 

14 

4.8 

8.5 

8.9 

657.4 

14.9 

21 

15 

4.6 

8.0 

7.8 

618.2 

9.2 

191 

21 

16 

4.9 

7.3 

8.3 

639,4 

14.5 

192 

22 

1 

5.6 

7.0 

8.0 

644,9 

14.7 

169 

22 

2 

4.7 

7.4 

8.1 

378.5 

34.8 

170 

22 

3 

7.5 

||8|^EWiVp 

576.8 

16.6 

171 

22 

4 

4.8 

9.4 

649.2 

172 

22 

5 

4.4 

7.7 

8.9 

617.5 

14.9 

173 

22 

6 

4.6 

8.0 

8.4 

656.9 

9.6 

174 

22 

7 

5.1 

7.5 

8.9 

616.1 

12.7 

175 

22 

8 

5.2 

7.5 

9.1 

649.1 

7.7 

176 

1  Maan  or  valuas  calculatad  for  pair*  or  observars  present  (weighted  by  the 
number  of  observers  Involvad).  Tha  mean  was  used  bacauia  valuta  for  dif¬ 
ferent  pairs  w art  In  good  agreement. 

a  Madlan  of  valuta  caleulatad  for  palra  of  obaarvara  praaant.  Tha  nttdlan 
was  usad  bacausa  occasional  largt  dlffarancaa  batwaan  pairs  occurrad  In 
the  data. 
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Performance  Characteristics  (Continued) 


Trial 

Round 

Time  to 
Signal 
Initiation 
(Sec) 

Audible 

Signal 

Duration 

£Sec) 

Visual 

Signal 

Ouratlon 

iSssl 

Height 

of 

Burst1 

(Feet) 

Deviation 

from 

Vertical2 

(Decrees) 

Firing 

Order 

23 

1 

4.4 

11.0 

10.0 

NR 

NR 

161 

23 

2 

4.6 

10.0 

6.0 

NR 

NR 

162 

23 

3 

4.9 

12.0 

7.0 

NR 

NR 

163 

23 

4 

4.5 

10.0 

7.0 

NR 

NR 

164 

23 

S 

4.8 

13.0 

8.0 

NR 

NR 

165 

23 

6 

4.6 

11.0 

7.0 

NR 

NR 

166 

23 

7 

4.1 

11.0 

8.0 

NR 

NR 

167 

23 

8 

4.7 

10.0 

7.0 

NR 

NR 

168 

T  Hein  oT  values  calculated  for  pair*  of  observers  present  (weighted  by  tin 
number  of  observers  Involved).  The  mean  was  used  because  values  for  dif¬ 
ferent  pairs  were  In  good  agreement. 

2  Median  of  values  calculated  for  pairs  of  obsarvers  present.  The  median 
was  used  because  occasional  large  differences  between  pairs  occurred  In 
the  data. 

NR  -  Not  recorded 
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APPENDIX  B 

Statistical  Analysis  of  Performance  Data 


1.  Summary.  Except  for  ona  cartridge  (trill  22,  round  2)  which  burst 
below  S00  feet,  all  cartridges  parfomad  a*  designed  ragardlast  of  tast 
conditions.  In  order  to  determine  whether  cartridge  performance  Was  sub¬ 
stantially  affected  by  changes  In  test  conditions,  however,  performance  data 
on  each  of  the  five  variables  were  analyzed  using  analysis  Of  variance.  No 
substantial  trends  were  discovered.  Nevertheless,  there  were  statistically 
significant  differences  for  each  variable  between  data  taken  under  different 
test  conditions.  In  particular,  the  cold  conditioned  rodnds  burst  signi¬ 
ficantly  lower  (40  feet)  and  significantly  farther  from  vertical  (6  degrees) 
than  the  average  round  while  rounds  from  the  first  box  of  cartridges  fired 
burst  significantly  higher  (45  feet)  and  significantly  closer  to  vertical  (3 
degrees)  than  the  average  round. 

2.  Linear  model  used.  Since  (except  for  the  first  eight  rounds)  cart- 
ridges  from  the  same  unit  pack  of  cartridges  were  always  subjected  to  the 
same  treatment  (e.g. ,  cold  conditioning)  and  fired  together  under  nearly 
identical  test  conditions,  any  trends  due  to  changes  In  test  conditions 
would  be  detectable  only  from  unit  pack  to  unit  pack.  Differences  In  per¬ 
formance  within  a  unit  pack  could  only  be  attributed  to  random  variation. 
Thus,  for  each  response  variable,  Y,  the  linear  model: 

V,k*M  *,8i  P,  *  t,k 

was  used  to  estimate  the  difference,  p,,  from  the  overall  mean,  p,  for  each 
of  the  n  unit  packs.  Least  squares  estimation  was  used,  and  the  p's  were 
subjected  to  the  side  condition: 

\il  ni  Pi  *  0 

where  n.,  was  the  number  of  available  observations  (for  the  reponse  variable 
under  consideration)  on  the  1in  unit  pack.  This  standard  paramatarlzatlon 
made  the  p-estlmators  Into  contrasts  and  tnablad  consideration  of  othar 
selected  contrasts  (In  particular,  that  for  cold  conditioned  rounds)  as 
linear  combinations  of  tha  p's. 

3.  Results.  Five  response  variables  were  analyzed  In  the  context  of  this 
linear  model :  time  to  signal  Initiation  In  saconds,  audible  signal  duration 
in  seconds,  visual  signal  duration  In  seconds,  height  of  burst  In  feet,  and 
deviation  from  vertical  In  dtgrtes.  In  addition,  halght  of  burst  and  devia¬ 
tion  from  vertical  were  analyzed  both  with  the  low  round  (trial  22,  round  2) 
Included  and  with  the  low  round  excluded.  An  analysis  of  varlanct  was 
performed  on  each  variable  (table  1),  and  the  F-value  for  testing  tha  null 
hypothesis  of  no  difference  In  performance  from  unit  pack  to  unit  pack  was 


greater  than  the  critical  F-value  for  0.005  significance  In  every  case.1 
Estimates  of  the  p's  (unit  pack  effects)  as  well  as  estimates  of  selected 
contrasts  found  to  be  of  interest  appear  In  table  2  along  with  the  calcu¬ 
lated  values  of  Student's  t  for  testing  whether  the  coefficients  are  zero. 

4.  Discussion.  Burst  locations  for  the  183  remotely  fired  rounds  are 
shown  to  seal e  In  figure  1.  Although  all  but  one  round  (a  cold  conditioned 
round)  burst  above  500  feet,  the  cold  conditioned  rounds  (even  excluding  the 
low  round)  tended  to  burst  significantly  lower  and  significantly  farther 
from  vertical  than  the  average  round.  However,  the  rounds  from  box  1 
(Including  the  Initial  8  rounds  and  24  additional  rounds  of  no 
particular  distinction)  tended  to  burst  significantly  higher  and  signifi¬ 
cantly  closer  to  vertical  than  the  average  round.  This  Indicates  that 
physical/chemical  differences  between  unit  packs  or  boxes  may  have  been 
significant  or  that  some  other  unmeasured  test  variables  (and  not  cold 
conditioning)  may  have  significance.  However,  none  of  the  observed  differ¬ 
ences  appeared  to  be  substantial.  There  were  no  clear  trends  with  tempera¬ 
ture  for  any  variable  (figures  2a  through  6a)  but  there  were  some  Indica¬ 
tions  of  trends  with  firing  order  (figure  2b  through  6b).  The  virtually 
constant  audible  signal  durations  measured  on  trial  23  (firing  order  160 
through  168;  these  were  all  of  the  rounds  fired  on  18  February)  seem  to  show 
merely  a  difference  In  measurement  technique.  Likewise,  the  trends  In  burst 
location  might  be  partially  due  to  day  to  day  variations  In  equipment  or  to 
meteorological  conditions  effecting  line  of  sight. 

1  The  assumptions  of  normality  and  homogeneity  of  variance  for  the  error 
terms  which  are  necessary  for  the  calculated  F-values  to  follow  the 
F-dlstrlbutlon,  were  not  satisfied.  In  particular,  It  Is  obvious  from  fig¬ 
ures  2  through  6,  that  times  measured  were  nearly  constant  for  some  unit 
packs,  and  the  distribution  of  the  residuals  for  degrees  deviation  from 
vertical  was  skewed.  Variable  transformation  and  sensitivity  analyses  on 
subsets  of  the  data  which  excluded  the  nearly  constant  times  could  have  been 
performed  but  were  not.;  It  was  felt  that  any  changes  In  the  results  obtained 
by  using  more  elaborate  analysis  would  be  minimal  and  would  not  justify  the 
cost  of  additional  analysis. 
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STATISTICAL  TESTING  OF  LARGE  COMPLEX  COMPUTER  SIMULATION  MODELS 

Carl  B.  Bates 

US  Army  Concepts  Analysis  Agency 
Bethesda,  Maryland  20814 

ABSTRACT >  In  response  to  Increasing  requirements  for  communications/ 
electronics  (EW)  analyses,  the  US  Army  Concepts  Analysis  Agency  (CAA) 
undertook  the  development  of  a  Divisional  Electronic  Warfare  Combat 
(DEWCOM)  Model.  The  model  has  been  developed,  and  It  contains  approxi¬ 
mately  20,000  lines  of  code  and  250  Input  variables.  It  simulates  two- 
sided  play  of  combat,  communications,  and  electronic  warfare  in  conven¬ 
tional  warfare  with  close  air  support.  Before  the  model  is  committed  to 
use  In  support  of  CAA  studies,  It  Is  undergoing  test  and  evaluation. 

The  test  and  evaluation  of  the  model  Is  being  conducted  In  two  phases— 
data  base  development  and  verification.  A  part  of  the  verification 
phase  Involved  a  sensitivity  analysis  of  model  output  to  changes  In 
model  Input.  A  group-screening  approach  was  applied,  and  a  resolution  V 
experimental  design  was  employed.  The  experimental  design  and  the  anal¬ 
yses  results  are  presented  and  discussed. 

1.  INTRODUCTION.  Due  to  Increasing  needs  for  the  performance  of  EW 
analyses,  CAA  undertook  the  development  of  an  EW  combat  simulation  mo¬ 
del.  Model  development  was  contracted  out}  the  developed  model,  Divi¬ 
sional  Electronic  Warfare  Combat  (DEWCOM)  Model,  was  delivered  to  CAA 
during  the  summer  of  1980.  The  model  Is  a  fully  computerized,  stochas¬ 
tic  simulation  model.  It  simulates  conventional  ground  warfare  with 
close  air  support.  Two-sided  play  of  combat,  communication,  and 
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electronic  warfare  Is  provided.  The  model  has  a  variable  force  resolu¬ 
tion  capability;  representation  of  up  to  five  echelons,  typically  corps 
to  company,  can  be  simulated.  The  model  Is  organized  Into  three  Inter¬ 
acting  modules  as  Illustrated  In  Figure  1.  One  module  performs  communi¬ 
cation  operations,  another  module  performs  tactical  operations,  and  the 
third  module  performs  electronic  warfare  operations.  The  modules  are 
driven  by  a  list  of  events  which  specify  actions  to  be  taken  at  a  sched¬ 
uled  time  In  the  future.  The  list  of  events  Is  Initialized  at  the  start 
of  the  simulation  by  the  user.  Thereafter,  the  list  Is  continuously  up¬ 
dated  as  a  consequence  of  actions  occurring  In  the  simulation.  This 
causes  additional  events  to  be  scheduled  for  future  combat,  communica¬ 
tion,  and  EW  operations.  The  process  continues  until  a  user  specified 
termination  time.  At  time  Intervals  specified  by  the  user,  model  re¬ 
ports  are  produced  summarizing  the  actions  which  have  been  simulated. 
Naturally,  before  the  model  could  be  approved  for  use  In  Agency  studies, 
the  model  had  to  be  tested.  Consequently,  a  test  and  evaluation  effort 
was  initiated. 

2.  TEST  METHODOLOGY 

a.  Objective.  The  objective  of  the  test  and  evaluation  was  to  es¬ 
tablish  that  the  model,  given  the  appropriate  Inputs,  accurately  repre¬ 
sents  the  performance  of  communications  and  EM  systems  In  a  tactical  en¬ 
vironment  and  portrays  realistic  combat  outcomes. 


The  test  and  evaluation  was  conducted  in  two  phases: 

1.  Data  base  development. 

i 1 .  Verification  testing  to  determine  (1)  if  the  various  functions 
in  the  model  performed  as  Intended  and  (2)  if  the  model  portrays  an 
accurate  representation  of  real-world  systems. 


Figure  1.  DEWCOM  Model  Organization 
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The  verification  proceeded  in  a  series  of  tests,  each  focusing  on  one 
or  more  of  the  module  operations.  Hand-calculated  results  were  compared 
with  appropriate  model  output.  Agreement  with  hand  calculations  consti¬ 
tuted  the  Part  I  verification  of  the  model  operation.  A  lack  of  agree¬ 
ment  was  attributed  to  either  Input  or  code  deficiency.  Both  possibili¬ 
ties  were  assessed,  and  Input  and/or  code  was  modified  and  the  model  re¬ 
run  If  necessary.  The  process  was  repeated  until  either  a  satisfactory 
model  output  was  achieved  or  until  enough  Information  was  obtained  that 
the  problem  could  be  addressed  at  a  later  date.  A  subset  of  the  results 
of  the  Part  I  verification  test  Is  Illustrated  In  Table  1. 

The  Part  II  verification  test  was  to  determine  if  the  model  gave  an 
accurate  representation  of  current  and  projected  combat,  communications, 
and  electronic  warfare  environment.  Part  II  consisted  of  three  sub¬ 
parts.  The  part  Involving  excursion  runs  from  the  base  case  and  the 
part  Involving  the  comparison  of  DEWCOM  with  another  model  are  not  dis¬ 
cussed  In  this  paper.  The  third  part  (sensitivity  test)  of  the  Part  II 
test  Is  discussed  in  the  remainder  of  this  paper. 
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Table  1.  Verification  Results 


TACTICAL 

Movement  Spurioni  movement  by  Cede  corrtcud  S 

EW  unlti'  Improper 
reference  to  uni t 
coordinate!  In  t«>t 
for  FEBA  dlitonco 


Direct  flri  attrition  Low  attrition  -  input  Alternate  algorithm  s 

diti  cal  1 bratton  undtr  contideration 

needed 

Indlroet  fir*  attrition  N*ar  tiro  attrition  Algorithm  ineorrtet  -  S 

rt placed 

Stop/reitart  Model  cannot  b*  proptrly  Remaini  to  b*  rnolvtd  U 

reitarted  if  change! 
affect  ongoing  activity 


COMMUNICATIONS 

Meuagi  genaration  Deicrlption  of  parameter!  Documentation  updatet  •  s 

controlling  menage  proparad 

generation  not  clear  ■ 
othenviie  aatitfaetory 

Meet  age  proceiting  Menage  proceited  only  by  Renal  ni  to  be  rnolved  M 

on*  net  type  •  no  routing 
over  other  net  typei  • 

•on*  redundant  neiiag* 
generation 

ELECTRONIC  WARFARE 

Locate  operation  Improperly  activated  ■  Cod*  corrected  S 

othemli*  aatitfaetory 

Intercept  operatloni  Improperly  activated  -  Code  corrected  S 

othenviie  tatlsfaetory 

Jam  operationt  Radar  algnal  not  Cod*  corrected  S 

Jammed  •  othenviie 
tattifactory 


NOTE i  S  ■  tatiifactory 
M  •  marginal 
U  ■  uniatltfactory 


3.  PROBLEM  DESCRIPTION  AND  BACKGROUND 


a.  A i •  The  purpose  of  the  sensitivity  test  was  to  identify 
those  Input  factors  which  have  the  largest  Impact  upon  selected  model 
output  variables  and  to  estimate  the  magnitude  of  the  Input  factor 
effects. 

b.  Background.  The  Concepts  Analysis  Agency  has  conducted  extensive 
computer  simulation  model  sensitivity  testing,  e.g.,  Bates  (1974), 

Thomas  (1975),  and  Bates  (1977).  Past  sensitivity  testing  has  always 
been  directed  at,  or  In  support  of,  a  particular  study;  that  Is,  a  study 
Involving  the  Investigation  of  tradeoffs  of  particular  combat  parameters 
which  could  be  associated  with  specific  model  Input  factors.  The  number 
of  Input  factors  to  be  Investigated  was  always  large  and  the  total  num¬ 
ber  of  computer  model  simulation  runs  was  always  limited.  In  all  cases, 
a  decision  had  to  be  made  between  the  number  of  factors  and  the  number 
of  factor  levels.  Invariably,  the  objective  was  to  Investigate  as  many 
Input  factors  as  possible.  Consequently,  ultimate  experimental  designs 
developed  were  2m  and/or  3n  fractional  factorial  designs. 

The  DEWCOM  sensitivity  test  was  different  from  previous  model  sensi¬ 
tivity  tests.  The  test  was  not  for  the  investigation  of  particular  In¬ 
put  factors  In  order  to  assess  the  applicability  of  the  model  for  a  par¬ 
ticular  study's  use.  The  test  was  a  part  of  an  overall  test  and  evalua¬ 
tion  of  the  model  following  its  Initial  development.  It  was  desired 
that  the  sensitivity  test  address  as  many  of  the  250  model  input  vari¬ 
ables  as  possible.  Ultimately,  a  group-screening  approach  was  used  In 
the  experimental  design  development. 


283 


c.  Group-screening  Designs.  Group-screening  experimentation  Is  not 
new.  Watson  (1961)  discusses  two-stage  screening  procedures.  Patel 
(1962)  and  LI  (1962)  Independently  Introduced  multistage  group-screening 
designs.  Hunter  and  Mezakl  (1964)  Illustrate  the  application  of  group- 
screening  designs  to  chemical  reaction  experimentation.  Kleljnen 
(1976a)  gives  a  Survey  of  screening  designs,  and  Kleljnen  (1975b) 
contains  a  more  detailed  discussion  of  screening  designs.  Mauro  and 
Smith  (1980)  examine  two-stage,  group-screening  methods,  and  Mauro  and 
Smith  (1981)  examine  a  random  balance/Plackett.-Burman,  two-stage  strat¬ 
egy. 

4.  EXPERIMENTAL  DESIGN.  A  detailed  examination  was  made  of  each  of  the 
250  model  Input  variables.  It  was  decided  to  consider  only  Blue  Input 
variables:  therefore,  the  Red  Input  variables  were  excluded.  Also  ex¬ 
cluded  were  variables  causing  abrupt  changes,  a.g.,  threshold  and 
switching  variables.  An  attempt  was  made  to  Include  only  those  vari¬ 
ables  having  a  continuous,  rather  than  discontinuous,  effect  upon  model 
output.  Also,  an  attempt  was  made  to  Include  variables  which  were  ex¬ 
pected  to  have  a  significant  effect  upon  model  output.  Ultimately,  ap¬ 
proximately  50  Input  variables  were  selected  for  Investigation.  Nominal 
values  were  then  selected  for  each  of  the  Input,  variables.  Finally,  a 
"high"  and  a  "low"  value  was  determined  for  each  of  the  variables.  Il¬ 
lustrations  of  the  Input  variables  and  their  values  are  given  In  Tables 
2,  3,  and  4.  Table  2  contains  tactical  variables,  Table  3  contains  com¬ 
munication  variables,  and  Table  4  contains  EW  variables.  The  high  and 
low  values  were  picked  to  be  those  values  which  were  expected  to  contain 
the  expected  achievable  within  the  1990  timeframe.  Care  was  taken  to 


defining  lows  and  highs  of  the  variables  being  grouped  together  so  that 
their  expected  effect  would  be  In  the  same  direction.  We  did  not  want 
variable  effects  to  Inadvertently  cancel  each  other. 


Table  2.  Tactical  Variables 


Taat  factor 

Input  variable 

Input  datcrlptlon 

Nominal 

value* 

Variation  about 
nominal  value 

n-i 

High 

unit 

TU. MOVE .RATE 

Unit  move  rata 

100  m/mln 

A  1/2 

X  2 

movement 

TU. RADIUS 

Circular  area  occupied  by  unit 

(100-DJ9)m 

X  1/2 

X  2 

D1 ract 
fire 

TU.MP.qUANTTTT 

Number  waapont  by  typa  owned 
by  untt 

(2-44) 

X  1/2 

X  2 

NT. COMBAT  .VALUE 

Combat  value  with  weapon  typa 

(2-100) 

X  4/5 

X  6/b 

DAMAGE. CLASS 

Coefficient  modeling  effects  of 
attrition  of  weapon  typa 

(60-100) 

♦25 

•25 

Indl ract 
fir* 

SO. ARTY. RESET, TIME 

Minimum  time  Interval  batwaan 
artillery  fire  missions 

2  min 

X  2 

X  1/2 

TU. SUPPRESSION. FACTOH 

Percent  decretal  In  unit 
effectfvenest  due  to  arty  fire 

(12-60)  min 

X  2 

X  1/2 

TU.DURATION.OF. SUPPRESSION 

Period  of  dtcrttse  In  unit 
effectiveness  due  to  arty  fir* 

(4-16)  min 

X  2 

X  1/2 

TU. ARTY. DURATION 

Duration  of  arty  fire  against 
unit 

15  min 

X  1/2 

X  2 

TU.ARTT, INTERVAL 

Interval  between  artlllary  fires 
against  unit 

1  min 

X  1/2 

X  2 

•Nominal  valuat  which  ihow  a  rang*  dnplct  the  spread  for  all  typaa  of  equipment  being  modeled,  *.g. ,  th*  combat 
values  aiiattid  each  toot  weapon  varied  from  2  to  100. 

Table  3.  Communication  Variables 


Test  factor 

Input  variable 

Input  datcrlptlon 

nominal 

value* 

Variation  about 
nominal  value 

Low  t  'Hgn 

Massage 

generation 


Hiiugi 

processing 


Network 
mil ntenene* 


SD. COORDINATION. INTERVAL 
CO.LENOTH 

BACKGROUND. TRAFFIC. UPDATE, TIME 
MAX. DEL AT 

CO. PROCESSING. TINE 

CE.FS.QUANTITT 
DAMAGE. CLASS 
CET.MTSF 

TU.COMH.SE TUP. TIME 
TU, COMM, TEARDOWN. TIME 


Minimum  tlm  bttwtun  matltgn  for 
units 

Trsnsmlsslon  tint  of  msttegt 


Interval  <t  which  traffic  delay* 
are  computed 

ManI nun  tin*  for  background 
truffle  delay 

Tin*  bifor*  and  (fttr  transmission 

nttdtd  to  precast  <  massage 

Initial  quantity  of  conwinlcatlon 
equipment 

Coefficient  modifying  affoeta  of  *i 
trltlon  on  cwewnleitlon  equlpmei 
Moan  tin*  batwaan  falluraa  for 
comunlcatlon  equipment 
Interval  naodad  to  aat  u»  wlra 
comunleatlon  for  unit 
'ntarvat  needed  to  taar  down  wlra 
comunleatlon  for  unit 


time 


30  ml 

X  2 

X  1/2 

(30-120)  sac 

X  2 

X  1/2 

15  min 

X  1/3 

X  3 

10  min 

X  2 

X  1/2 

(1-3)  min 

X  3 

X  1/3 

(1-6) 

X  1/2 

X  6/5 

t-  100 

+25 

-25 

(600-200)  hr* 

X  1/2 

X  2 

(6-10)  min 

X  2 

X  1/2 

(5-10)  min 

X  2 

X  1/2 

ling  modeled,  t.g. 
minutes. 

i ,  the 

processing 
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Table  4.  Electronic  Warfare  Variables 


Nominal 

Variation  sbout 

Test  factor 

Input  variable 

Input  description 

value* 

nominal  va'uu 

^^Lowljllilt^ 

Intercept 

EWK.FS.QUAMTI  TV 

Initial  quantity  of  Intercept 
equipment 

Coefficient  modifying  effects  of 

1 

X  1/2 

X  2 

DAMAGE .CLASS 

100 

+2b 

-26 

1NT.MTHF 

attrition  on  Intercept  equipment 
Mean  time  between  failure  for 

70  hrs 

X  1/2 

X  2 

ENT.MTTN 

Intercept  equipment 

Mean  time  to  repair  for  Intercept 

3  hrs 

X  2 

X  1/2 

Locate 

ENT. DF. TIME 

equipment 

Time  period  (In  min)  to  perform 

UF  function 

Percent  decrease  In  Intel  rate 

(1-3)  min 

X  3 

X  1/3 

SD.DF.RATE.l, UNIT. OUT 

sot 

♦26 

-26 

SO. DF, RATE .2. UNITS. OUT 

for  DF  less  1  station 

Percent  decrease  In  Intel  rate 

711 

♦26 

-26 

ENE.FS.qUANTITV 

for  DF  less  2  stations 

Initial  quantity  of  locate 

1 

X  1/2 

X  2 

DAMAGE . CLASS 

equipment 

Coefficient  modifying  effects  of 

10U 

♦26 

-26 

ENT.MTBF 

attrition  on  locate  equipment 

Mean  time  between  failures  for 

70  hrs 

X  1/2 

X  2 

ENT.MTTN 

locate  equipment 

Mean  time  to  repair  lonate 

3  hrs 

X  2 

X  1/2 

Qround 

ENE.FS.QUANTITV 

equipment 

Initial  quantity  of  surveillance 

1 

X  1/2 

X  2 

surveillance 

DAMAGE . CLASS 

equipment 

Coefficient  modifying  offects  of  at¬ 
trition  on  surveillance  equipment 
Mean  time  between  failures  for 

too 

♦26 

•26 

ENT.MTBF 

70  hrs 

X  1/2 

X  2 

CNT.MTTK 

surveillance  equipment 

Mean  time  to  repair  for 

.1  hrs 

X  2 

X  1/2 

ENT.PONER 

surveillance  equipment 

Power  output  of  surveillance 

4U  db 

-20 

♦  60 

equipment 


‘Nominal  va Iuqs  which  show  I  range  depict  the  ipread  for  til  types  of  equipment  being  modeled,  e.y , ,  the  time  to 
perform  the  DF  function  for  tho  varloue  EN  equipment  varied  from  I  to  3  mlnutei. 


. . lum  «m*uiw‘ww 


Simultaneous  to  the  above,  the  primary  operations  of  each  module  were 
enumerated  and  basic  military  functions  were  associated  with  the  module 
operations.  The  association  between  the  module  operations  and  military 
functions  Is  shown  In  Table  5.  The  11  module  operations  are  hereafter 
termed  model  Input  factors.  The  experimental  design,  therefore,  In¬ 
volved  the  11  two-level  Input  factors.  All  factors  are  completely 
crossed.  It  was  suspected  that  the  factors  within  a  module  may  Inter¬ 
act.  Consequently,  an  experimental  design  which  would  permit  assessment 
of  main  effects  and  first  order  (two-factor)  Interactions  was  desired. 

A  1/16  x  2^  experiment  was  designed  using 

I  -  ABEFJL  •  CDEFKL  ■  BCEGJKL  ■  ABCOEFGH 

as  the  defining  contrast.  The  fractional  factorial  design  required  128 
model  runs  and  permitted  assessment  of  the  11  main  effects  and  the 
(*g)  ■  55  two-factor  Interaction  effects. 

The  analysis  of  variance  (ANOVA)  model  for  the  design  Is 

y-/U  +  A  +  8  +  ...  +  L+AB+AC  +  ...  +KL, 

where  /u  Is  a  true  but  unknown  effect;  A,  B,  ...,  L  are  factorial  ef¬ 
fects;  and  y  Is  a  particular  model  output  variable. 
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Table  5.  Test  Factors 


Ml  1 Itary  functions 

Test 

DEWCOM 

Ind 

Cmd 

Intel 

factor 

operations 

Mvr 

fire 

&  con 

Comm 

coll 

EW 

TACTICAL  MODULE 

A  Unit  movement  X 

B  Direct  fire  X 

C  Indirect  fire 

COMMUNICATIONS  MODULE 
D  Message  generation 

E  Message  processing 

F  Network  maintenance 

EW  MODULE 
G  Intercept 

H  Locate 

J  Ground  surveillance 

K  Intel  acquisition 

L  Jamming 


X 


X 

X 


X 

X 

X 

X 


The  following  four  measures  of  effectiveness  (MOE)  were  selected  as 
model  output  variables  for  analysis; 

Red  personnel  losses 
Red  weapons  losses 
Blue  personnel  losses 
Blue  weapons  losses 

The  simulation  experiment  was  executed  In  accordance  with  the  experimen¬ 
tal  design,  and  the  four  MOE  were  analyzed  In  accordance  with  the  analy¬ 
sis  plan. 
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5.  ANALYSIS.  The  analysis  of  variance  model  Is  a  fixed  effects  model. 
Consequently ,  all  11  main  effects  and  all  55  Interaction  effects  in  the 
ANOVA  table  are  tested  over  the  Mean  Square  (Residual)  which  has  61 
degrees  of  freedom.  For  example,  If  MS(A)/MS(R)  is  equal  to  or  greater 
than  Fj  (l-aO*  ^nPut  factor  A  is  statistically  significant  at  the 
<*-1evel  of  significance.  For  each  MOE,  the  marginal  and  two-way  means 
were  tabulated.  ANOVAs  were  performed  and  the  marginal  means  and  sig¬ 
nificant  Interactions  were  plotted. 

The  ANOVA  results  are  summarized  In  Table  6,  First,  considering  the 

two  Red  MOE,  we  see  that  three  Interactions,  OE,  EF,  and  FK,  are  sig¬ 

nificant  for  Red  personnel,  and  two  Interactions,  DE  and  FK,  are  sig¬ 
nificant  for  Red  weapons.  Each  MOE  has  the  same  significant  main  ef¬ 
fects— A,  B,  F,  and  K.  The  decreasing  order  of  the  four  significant 

main  effects  was  B,  K,  F,  and  A  for  both  Red  MOE.  Also,  the  direction 

of  the  effects  was  as  expected--Red  losses  decrease  as  the  Input  factors 
are  changed  from  low  to  high  levels.  An  examination  of  the  Blue  MOE  re¬ 
sults  shows  that  the  seven  significant  main  effects  are  a  subset  of  the 
significant  Interaction  effects.  In  addition,  the  significant 
Interactions  contain  factors  F  and  K.  Therefore,  all  Input  factors  ex¬ 
cept  G  and  J  have  a  significant  influence  upon  both  Blue  MOE.  Expecta¬ 
tions  were  that  changing  the  Input  factors  from  low  to  high  would  have 
an  Increasing  effect  upon  both  Blue  MOE;  however,  the  change  from  low  to 
high  of  factors  B,  0,  and  E  had  a  decreasing  effect.  This  apparent  ano¬ 
maly  was  explained  after  subsequent  study  of  the  model  and  Input. 


6.  SUMMARY.  Group-screening  designs  have  potential  application  for  the 
statistical  testing  of  large  complex  computer  simulation  models.  How¬ 
ever,  to  date,  the  literature  seems  void  of  illustrations  of  group¬ 
screening  designs  applied  to  large  real-world  simulations.  The  above 
Illustrates  only  the  first  stage  In  the  application  of  group  screening. 
Subsequent  stages  are  essential  for  the  usefulness  of  group-screening 
designs  to  be  realized.  For  the  above  described  problem,  possibly  reso¬ 
lution  III  designs  should  be  used  rather  than  resolution  V  designs.  Be¬ 
cause  of  the  large  number  of  interactions  In  the  above  Illustration, 
however,  It  appears  that  resolution  III  designs  would  be  Inappropriate 
even  for  early  stages.  Too  much  care  cannot  be  taken  In  the  grouping  of 
factors  to  ensure  that  effects  do  not  cancel  each  other.  Also,  the  di¬ 
rection  of  the  effect  of  grouped  factors  must  be  known  to  be  the  same. 
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SELLING  A  COMPLICATED  EXPERIMENTAL  DESIGN 
TO  THE  FIELD  TEST  OPERATOR 

Carl  T.  Russell 

US  Army  Cold  Regions  Test  Center 
Fort  Greely,  Alaska 

ABSTRACT.  After  studying  the  objectives  and  planning  constraints  for 
cold  regionf  performance  testing  with  the  main  gun  of  the  XM1  tank,  the 
author  determined  that  a  rather  complicated  experimental  design  seemed 
appropriate.  In  particular,  the  blocking  scheme  required  thot  carefully 
chosen  quarter  replicates  of  a  24  design  be  conducted  sequentially.  Such  a 
design  lays  out  a  firing  schedule  almost  round  by  round  and,  on  the  surface, 
presents  the  test  operator  with  Insurmountable  problems  In  execution.  A 
test  operator  generally  expects  and  receives  only  Instructions  to  complete  a 
prescribed  number  of  ''replications''  under  each  combination  of  test  condi¬ 
tions,  and  he  deeply  resents  the  Intrusion  of  a  statistician  Into  detailed 
test  scheduling.  By  carefully  emphasizing  the  practical  and  Intuitively 
advantageous  aspects  of  the  experimental  design,  however,  the  author  was 
able  to  sell  the  design  to  the  test  operator,  and  the  sales  pitch  is  the 
topic  of  this  paper. 

I.  INTRODUCTION  AND  STATEMENT  OF  THE  PROBLEM.  Winter  temperatures  be¬ 
low  freezing  are  common  In  highly  Industrialized  regions  of  Europe,  Russia, 
Red  China,  and  Korea.  Since  these  are  all  regions  where  the  U.S.  Army  could 
fight,  the  need  for  a  test  center  to  examine  the  performance  of  Army  person¬ 
nel  and  materiel  In  the  cold  Is  obvious.  The  U.S.  Army  Cold  Regions  Test 
Center  (CRTC)  Is  located  at  Fort  Greely,  Alaska,  where  temperatures  are 
below  32°F  more  than  80  percent  of  the  time  during  the  winter  months,  aver¬ 
age  49  days  per  year  below  -25#F,  and  average  an  annual  low  of  -59°F. 
Although  colder  areas  exist  than  Fort  Greely,  no  other  accessible  area  In 
the  United  States  Is  available  to  the  U.S.  Army  for  cold  regions  testing  of 
military  systems. 

As  part  of  Development  Test  II  of  the  XM1  tank  (Ml  Abrams  tank),  cold 
regions  testing  was  conducted  at  CRTC.  Main  gun  firing  performance  at  tem¬ 
peratures  below  0°F  was  one  of  the  many  Issues  to  be  addressed  during  this 
testing.  Criterion  values  for  probability  of  hit  against  targets  of  pre¬ 
scribed  size  were  given  In  a  matrix  for  each  combination  of  four  test  fac¬ 
tors: 

•  Tank  Mode— stationary  or  moving, 

-  Target  Mode-stationary  or  moving, 

•  Nominal  Range— short  (approximately  1500  meters)  or  long  (approxi¬ 
mately  2500  meters), 

•  Round  Type— high  explosive  anti-tank  (HEAT)  or  kinetic  energy  (KE). 
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In  addition,  there  were  requirements  to  compare  firing  performance  of  the 
XM1  with  the  standard  M60  tank  and  to  determine  whether  XM1  firing  per* 
formance  degraded  at  lower  temperatures. 

There  were  numerous  constraints: 

•  Rounds  were  to  be  fired  In  five-round  shot  groups  at  panel  targets 
with  hit  probabilities  to  be  estimated  from  Impact  coordinates  for 
each  five-round  group. 

•  Around-the-clock  testing  was  planned,  with  firing  periods  sandwiched 
between  mobility  exercises.  Four  shot  groups  per  firing  period 
appeared  reasonable  and  feasible,  and  three  tank  craws  ware  avail¬ 
able  for  test  conduct. 

•  Weather  conditions  were  uncontrolled.  Temperature  was  a  factor  of 
direct  Interest,  but  other  factors  (such  as  visibility)  were  regard¬ 
ed  primarily  as  nuisance  factors. 

•  An  Important  decision  point  was  scheduled  before  test  termination, 
so  partial  data  had  to  be  Interpretable. 

•  Although  the  criterion  addressed  only  HEAT  and  KE  rounds,  two  types 
of  KE  rounds— armor  piercing  descardlng  sabot  (APDS)  and  armor  pierc¬ 
ing  fin  stabilized  discarding  sabot  (APFSDS)— were  provided  for 
test. 

•  A  few  high  explosive  projectile  (HEP)  rounds  were  also  provided  for 
test,  and  there  was  some  Interest  In  ranges  other  than  1500  and  2500 
meters. 

Previous  test  planning  had  Identified  the  number  of  rounds  to  be  fired, 
and  an  unbalanced  test  matrix  (Table  1)  had  been  formulated  to  spread  the 
rounds  over  the  test  conditions.  This  matrix— seldom  differentiated  from 
the  test  design— Is  typical  of  those  usually  proposed  for  field  tests,  and 
It  would  typically  be  analyzed  as  If  It  were  conducted  as  a  completely 
randomized  experimental  design.  But  Its  conduct  would  almost  certainly  have 
been  dictated  by  efficiency,  resulting  In  little  actual  randomization. 

Russell  (2)  argued  that  because  completely  randomized  designs  are  Inim¬ 
ical  to  efficient  test  conduct  In  a  field  environment  (they  require  overall 
conduct  by  chance  rather  than  by  organization),  they  should  be  replaced 
wherever  possible  by  designs  requiring  only  small-scale  randomization  easily 
generated  during  day-to-day  conduct.  An  obvious  approach  was  advocated: 
design  In  blocks  compatible  with  test  constraints  and  executable  within  a 
relatively  short  time  period,  repeating  similar  blocks  throughout  the  test. 
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TABLE  1:  Tentative  Test  Matrix  (Tabulated  Values  Are  Number  of 
Five-Round  Shot  Groups  at  Each  Combination  of  Test  Conditions) 


Tank/Target 

Round 

XMl  Below 

w 

"7M1  A'bo've 

M€0  Above 

■=25^ 

Mode 

Type 

1500 

250o 

1500 

2500 

‘  1500" 

2500 

HEAT 

4 

4 

5 

5 

3 

3 

S/S 

APDS 

5 

5 

7 

7 

4 

4 

APFSDS 

4 

4 

6 

6 

3 

3 

, 

HEP 

2 

2 

4 

4 

A 

L 

2 

HEAT 

4 

4 

4 

4 

3 

3 

S/M 

APDS 

5 

5 

5 

5 

4 

4 

APFSDS 

4 

4 

4 

4 

3 

3 

HEAT 

4 

4 

4 

4 

3 

3 

M/S 

APDS 

5 

5 

5 

5 

4 

4 

APFSDS 

4 

4 

4 

4 

3 

3 

HEAT 

4 

4 

4 

4 

3 

3 

M/M 

APDS 

5 

5 

5 

5 

4 

4 

APFSDS 

4 

4 

4 

4 

3 

3 

I  used  this  "basic  matrix  approach"  to  devise  a  rather  complicated 
revision  to  the  tentative  design  for  XM1  firing  performance  which  essential¬ 
ly  scheduled  the  crew  and  order  for  every  shot  group  fired.  Fundamentally, 
the  complication  was  that  Inherent  In  any  statistically  sound  field  test 
design:  Instead  of  emphasizing  sample  size  In  terms  of  requisite  "replica¬ 
tions"  In  each  cell  (with  an  unrealistic  request  for  complete  randomiza¬ 
tion),  the  statistical  advice  concentrated  on  a  method  of  detailed  test  con¬ 
duct  directed  at  obtaining  a  data  set  amenable  to  thorough  statistical  anal¬ 
ysis.  From  the  point  of  view  of  a  test  operator  (at  CRTC,  these  are  usually 
0-3's),  such  detailed  statistical  advice  Is  Inherently  unwelcome.  Instead 
of  setting  e  clear  objective  (get  so  many  observations  per  cell)  with  what 
Is  precelved  as  minimal  guidance  (randomize),  the  advice  appears  to  set  a 
vague  objective  (get  a  good  data  set)  with  strangling  guidance  (da  it  just 
this  way).  Thus  my  problem  evolved  from  creating  a  sound  design  to  selling 
It.  How  could  1  convince  the  test  operator  that  It  was  possible  and  advan¬ 
tageous  to  execute  my  proposed  design  rather  than  simply  obtain  required 
"replications"  of  cells  In  some  matrix? 
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My  solution  was  to  prepare  and  present  a  briefing  designed  to  show  not 
only  that  my  proposed  design  was  executable  but  also  that  It  provided  organ¬ 
ized  solutions  to  potential  problems  of  test  conduct  while  leaving  a  great 
deal  of  flexibility  for  the  test  operator  and  relieving  him  of  awkward  plan¬ 
ning  details.  The  next  section  of  this  paper  presents  this  briefing  In 
narrative  form.  A  brief  technical  discussion  of  the  design  follows,  and  the 
paper  concludes  with  a  few  final  comments  and  a  summary  of  the  test  outcome. 

II.  THE  SALES  PITCH.  The  briefing  consisted  of  six  parts:  an  Intro- 
ductlon  stating  the  design  goals,  an  overview  of  the  design  which  described 
the  design  In  terms  of  four  prioritized  test  matrices,  and  discussions  of 
each  test  matrix  In  order.  Because  the  briefing  was  a  sales  pitch,  it 
emphasized  In  nontechnical  terms  why  my  proposed  design  should  be  conducted, 
how  It  could  be  conducted,  and  how  It  would  provide  advantages  to  the  test 
operator  which  at  least  offset  Its  disadvantages. 

A.  Experimental  Design  Goals.  This  portion  of  the  briefing  told  what 
I  was  trying  to  accomplish: 

•  Compare 

-stationary  versus  moving  tank 
-stationary  versus  moving  target 
-HEAT  versus  KE  (ARCS  and  APFSDS)  rounds 
-1500  meter  versus  2500  meter  tank/target  range 
-XM1  versus  M60 

under  test  conditions  as  similar  as  possible. 

•  Make  these  companions  over  as  wide  a  variety  of  test  conditions  as 
possible,  but  do  so  In  such  a  way  that  the  effects  of  selected  test 
conditions  (In  particular  temperature)  can  probably  be  Isolated. 

•  Preplan  order  of  trials  In  such  a  way  that  as  much  balance  as  pos¬ 
sible  Is  maintained  on  a  day-to-day  basis. 

-to  Increase  the  likelihood  that  reasonably  accurate  partial 
results  will  be  available  quickly. 

-to  minimize  the  Impact  of  unforeseen  delays. 

•  Allow  sufficient  flexibility  that,  with  a  reasonable  amount  of  good 
luck,  the  design  can  be  executed. 

B.  Overview  of  Proposed  Design.  This  portion  of  the  briefing  describ¬ 
ed  the  overall  test  In  terms  of  four  test  matrices  (Figure  1):  two  test 
matrices  for  XM1  alone  (one  for  each  temperature  range  of  Interest),  a 
matrix  for  the  XM1  versus  M60  comparison,  and  a  matrix  for  side  tests  and 
make-up.  The  emphasis  here  was  on  overall  resource  distribution  rather  than 
detailed  test  structure.  Together  with  Table  2,  which  compared  the  test 
matrix  associated  with  the  proposed  design  to  the  tentative  matrix  of  Table 
1,  Figure  1  was  meant  to  reassure  the  audience  that  no  radical  departure 
from  the  status  quo  was  being  advocated.  But  both  Figure  1  and  Table  2  were 
also  used  to  point  out  two  Inherent  advantages  of  the  proposed  design, 
namely  Its  balance  (In  my  experience  balance  appears  to  almost  anyone  as 
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TABLE  2:  Comparison  of  the  Proposed  Test  Matrix  to  the  Tentative  Test  Matrix  of  Table  1 
(Tabulated  Values  Are  Nuaber  of  Five-Bound  Shot  Groups  at  Each  Combination  of  Test  Conditions) 


XM1  ABOVE  -25*  F  XM1  BELOW  -25*  F 


an  mat 


126  HKAT 

120  mm 

120 


TSTfiKF 

3M  mm  120  mm 

_ — JM 


XM1  VERSUS  M60  COMPARISON  (ABOVE  -25*  F) 

- igiri - rarnsr” 

XM1,  320  ItDk  120  ATC2  j  M2Q.  320  MM  120  AR02 

_ _ 120  mnoa  , _ no  mmm 


XM1  SIDETESTS  it  MAKE-UP 

- ItfMi"' 

2io  mot  40  mum 

20  HP 


Figure  1.  Overall  Structure  of  Proposed  Design  In  Terms  of 
Resourcss  Distributed  Among  Four  Test  Motrleee* 


Intuitively  advantageous)  and  Its  formal  dlitlnctlon  between  hlghait  prior¬ 
ity  taitlng  (XM1  alone,  •ipaclally  abova  -25°F),  lacondary  taitlng  (XM1 
versus  HSO),  and  testing  to  be  done  If  possible.  A  problem  with  three  crews 
was  also  discussed  connection  with  balance.  It  would  clearly  be  desirable  to 
have  each  crew  fire  the  same  number  of  five-round  shot  groups  under  similar 
conditions,  but  three  crews  cannot  possibly  fire  cell  totals  of  two,  four, 
five,  or  seven  shot  groups  (from  the  tentative  matrix)  In  a  balanced  fash¬ 
ion.  Conveniently,  my  revised  design  requires  exactly  three  shot  groups  In 
every  cell  except  those  In  the  XM1  side  test. 

C.  XM1  Trials  Above  -25”F.  This  portion  of  the  briefing  discussed  the 
details  of  the  test  design  for  the  highest  priority  test  matrix  In  depth. 

A  "basic  matrix"  of  test  conditions  (Table  3)  was  Introduced  and  terms 
were  defined.  Each  combination  of  conditions  In  the  matrix  (cell)  was  to  be 
executed  three  times,  once  by  each  crew.  One  execution  of  a  cell  (mission) 
was  to  consist  of  a  crew  firing  a  five-round  shot  group  under  tna  stated 
conditions.  A  trial  was  to  consist  of  four  prescribed  missions  by  the  same 
crew  during  oneTTrTng  period. 

The  key  to  the  design  was  the  typical  trial.  A  specific  example  of  the 
typical  trial  was  given  for  crew  3: 

S/S,  HEAT,  1500 
S/M,  HEAT,  2500 
M/S,  APDS,  2500 
M/M,  APOS,  1500. 
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TABLE  3*.  Basic  Test  Matrix  (Calls  Numberad  for  Rafaranca) 


Tank/Target 

HEAT 

APDS 

APDSDS 

Mode 

1500 

2500 

1500 

2500 

1500 

2500 

S/S 

1* 

2 

3 

4 

5 

6 

S/M 

7 

8* 

9 

10 

11 

12 

M/S 

13 

14 

15 

16* 

17 

18 

M/M 

19 

20 

21* 

22 

23 

24 

*Th1s  Is  tha  spaclflc  example  of  a  "typical  trial". 


Tha  typical  trial  was  than  dlscussad  both  In  terms  of  this  spaclflc  axampla 
and  a  ganaral  varslon  which  spaclflad  that  a  craw  was  to  axacuta  four  mis¬ 
sions: 

•  Ona  at  aach  tank/targat  moda, 

•  Two  with  aach  of  two  round  types, 

•  Each  round  typa  at  both  rangas. 

(Tha  ordar  of  missions  within  aach  trial  was  to  ba  randomlzad  as  much  as 
posslbla  within  tast  constraints.) 

Tha  dlsadvantaga  of  trials  of  this  sort  was  openly  admlttad:  tast 
conditions  within  avary  trial  wars  to  ba  totally  mlxad  with  nothing  flxad. 
Craws  would  not  ba  allowad  to  flra  aaslar  targats  (that  Is,  stationary  tank 
or  targat,  short  ranga)  first,  tha  moving  targat  would  hava  to  ba  aval  labia 
for  avary  trial,  and  caraful  Instruction  of  craws  would  ba  nacassary  to  pre- 
vant  round  typas  balng  flrad  at  tha  wrong  tank/targat  modas  or  tha  wrong 
rangas.  On  tha  surfaca,  for  Instanca,  using  tha  sams  round  typa  and  firing 
only  ona  targat  throughout  a  trial  would  ba  mora  officiant  and  lass  prona  to 
arror  than  trials  with  tha  proposed  structure.  But  tha  great  advantage  of 
trials  with  tha  proposed  structure  was  also  pointed  out:  tha  tast  opera¬ 
tor's  tally  sheet  would  ba  nearly  balanced  after  aach  trial  with  the  same 
number  of  missions  at  each  range,  at  aach  targat/tank  moda,  and  (as  much  as 
ona  could  hope)  at  aach  round  typa.  Furthermore,  although  firing  of  easier 
targets  first  was  not  universally  permitted,  It  was  stressed  that  no  great 
harm  would  ensue  If  In  soma  of  tha  earlier  trials  aaslar  targets  ware  fired 
first.  Likewise  a  certain  amount  of  systematic  manipulation  of  ranges  was 
permissible  so  long  as  It  was  not  always  done  the  same  way:  for  Instanca, 
In  tha  specific  example,  the  tank  could  fire  first  on  the  move  five  APDS 
rounds  against  a  2500  meter  stationary  target,  than  fire  stationary  HEAT 
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rounds  against  a  2500  meter  moving  targat,  followed  by  tha  missions  against 
1500  mater  targets. t 

After  this  somewhat  lengthy  discussion  of  tha  typical  trial,  tha  pro¬ 
posed  order  for  conduct  of  trials  was  given  In  terms  of  prioritized  lists  of 
trials  for  each  crew  (Table  4).  Tha  specific  typical  trial  discussed  previ¬ 
ously  was  Identified  In  the  table,  and  a  brief  examination  of  other  trials 
showed  that  they  are  Indeed  very  similar. 

TABLE  4:  Prioritized  Lists  of  Trials  for  Each  Crew,  XM1  Alone, 

Above  -25°F  (Numbering  from  Table  3) 


Crew  1 

2,  7,15,22 

6.11.13.20 
1,  8,18,23 

3,10,14,19 
4,  9,17,24 

5.12.16.21 


Crew  2 

3.10.14.19 

3,  8,18,23 

6.11.13.20 
2,  7,15,22 

5.12.16.21 

4.  9.17.24 


Crew  3 

1,  8,16,21* 
5,12,14,19 
2,  7,17,24 
4,  9,13,20 

6,11,15,22 
3.ilOJt&i33 


*Th1s  Is  the  specific  example  of  a  "typical  trial". 


The  Idea  behind  the  lists  was  as  follows.  If  after  a  mobility  run  the 
temperature  were  between  0°F  and  -25°F,  whatever  crew  was  In  the  tank  would 


tAs  anticipated,  this  cavalier  attitude  towards  randomization  raised  com¬ 
ments  following  the  paper's  formal  presentation.  What  I  was  trying  to  give 
the  test  operator,  however,  was  an  executable  design  with  reasonable  flexi¬ 
bility.  In  fact,  1  eventually  provided  the  test  operator  with  preferred 
order  of  conduct  In  the  form  of  mlsslon-by-mlsslon  schedule  lists,  and 
mission  order  In  these  lists  was  formally  randomized  within  trial.  But  I  do 
not  believe  that  minor  nonsystematlc  deviations  from  a  formal  randomization 
scheme  could  markedly  effect  the  experimental  results  In  an  experiment  of 
this  size,  and  I  doubt  that  even  naively  systematic  deviations  from  within 
trial  randomization  could  have  overwhelming  effects.  Randomization  within 
trials  would  protect  primarily  against  possible  tendencies  for  crew  perfor¬ 
mance  to  vary  consistently  within  trials  rather  than  from  trlal-to-trlal. 
Although  attempting  such  randomization  Is  worthwhile,  Its  benefits  should  be 
small  unless  consistent  wlthln-trlsl  trends  are  substantial  relative  to 
effects  of  primary  Interest— which  I  believe  to  be  highly  unlikely  In  the 
present  case.  Thus  even  naively  systematic  wlthln-trlal  ordering  should 
produce  only  small  bias  under  reasonable  assumptions  about  operational 
performance.  In  fact,  conducting  a  test  In  small  blocks  of  time  and  space 
forces  any  potential  damage  from  nonrandomization  to  be  small  (under  mild 
assumptions),  permitting  the  test  operator  substantial  deviations  from 
formal  randomization  within  blocks.  To  me,  the  risk  of  bias  from  lack  of 
wlthln-trlal  randomization  Is  small  compared  to  the  risk  of  losing  Influence 
over  test  conduct  by  pedantically  restraining  the  test  operator. 
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fire  Hi  naxt  scheduled  trial,  (If  th«  temperature  were  between  -2S°F  and 
-50#F,  tha  craw  would  flra  Its  naxt  schadulad  trial  from  t  similar  list  for 
trials  balow  -25°F,  and  If  tha  tamparatura  wara  abova  0*F  or  balow  -50*F,  no 
firing  would  taka  pi act. )  Craws  would  than  change,  and  anothar  mobility  run 
would  bagln. 

With  raasonabla  attention  to  craw  scheduling  and  soma  luck.  It  should 
ba  possible  to  conduct  at  laast  thraa  or  four  trials  by  each  craw  according 
to  tha  proposad  ordar,  Toward  tha  and  of  tha  test,  howavar,  Instances  could 
ba  axpactad  whan  tha  craw  In  tha  tank  had  alraady  complatad  all  Missions  In 
tha  tamparatura  ranga  prasant  at  tha  and  of  a  mobility  run.  In  such  In¬ 
stances,  tha  tast  operator  was  advised  simply  to  have  tha  craw  flra  (from 
bottom/up  In  tha  list)  any  available  mission  for  anothar  craw  In  tha  correct 
tamparatura  range*  If  no  such  trials  In  tha  correct  tamparatura  range  wara 
available,  than  based  on  tast  time  remaining,  available  trials,  ana  fore¬ 
casted  weather,  tha  tast  opsrator  could  opt  not  to  firs  or  opt  to  flra  an 
available  trial  from  a  list  for  tha  Incorrect  tamparatura  ranga  (working 
from  bottom/up  In  tha  list,  preferably  using  tha  correct  craw).  Likewise, 
If  tha  moving  target  array  should  break  down,  tha  tast  operator  was  advised 
simply  to  flra  (from  bottom/up)  tha  first  available  trial.  Ignoring  tha  re¬ 
quirement  for  moving  targets;  that  Is,  fire  all  four  missions  at  stationary 
targets  but  use  tha  tank  mode,  ranga,  and  round  type  specified,  Other  oper¬ 
ational  problems  wara  pqrtrayed  slmllarlly: 

•  If  all  goes  wall,  conduct  tha  naxt  available  trial  from  top/down 
In  the  prioritized  list  for  ths  crew  In  tha  tank. 

•  If  problems  arise  but  a  decision  to  firs  anyway  Is  prudent,  conduct 
tha  lowest  priority  available  trial  In  as  close  accordance  with  the 
prescribed  conditions  as  possible. 

•  The  statistician  would  ba  available,  fat  any  time  to  provide  advice. 

A  field  tast  Is  a  moving  train,  and  the  engineer  deserves  advice  which  will 
help  him  ba  on  time. 

Tha  proposed  ordering  of  trials,  If  executed  as  Just  described,  pro¬ 
vides  tha  statistician  with  a  usable  data  sat  even  If  many  trials  cannot  ba 
complatad  according  to  plan.  In  fact,  provided  there  wara  no  great  differ¬ 
ence  In  firing  performance  batwaan  APOS  and  APFSDS  rounds  (none  was  axpact¬ 
ad): 

•  Once  any  thraa  of  tha  first  four  trials  with  any  one  craw  were  com¬ 
pleted,  tha  data  sat  would  ba  usable, 

•  Once  tha  first,  second,  third,  or  fourth  trials  on  all  three  craws 
wara  completed,  tha  data  set  would  ba  usable. 

If  only  half  tha  data  wara  obtained  In  accordance  with  the  prescribed  plan, 
tha  statistician  would  ba  In  pretty  good  shape  for  analysis  (but  his  statis¬ 
tical  statements  could  not  baas  precise  as  with  a  complete  data  sat),  Some 


Intuitive  understanding  of  why  thli  It  to  can  be  teen  by  examining  how  the 
basic  matrix  fills  up  trial-by-trial  (Figure  2).  All  cells  of  the  HEAT 
versus  KE  matrix  fill  up  In  an  organized  way  as  trials  progress,  with  at 
least  one  observation  per  cell  after  the  first  two  proposed  trials  for  each 
crew  and  with  three  observations  per  cell  (one  for  each  crew)  after  the 
first  four  proposed  trials  for  each  crew.  The  last  two  trials  for  each  crew 
compare  only  the  two  KE  round  types,  filling  In  the  holes  left  after  the 
first  four  trials.  (The  demonstration  In  Figure  2  was  accomplished  with 
overlaid  vu-graphs  In  the  actual  briefing.) 

The  test  operator  would  have  to  devise  some  sort  of  organized  schedule 
even  to  fill  the  tentative  matrix  of  Table  1.  What  the  statistician  has 
done  here  Is  to  relieve  the  test  operator  of  a  tedious  task  by  providing  him 
with  a  balanced  version  of  Table  1  together  with  a  flexible  schedule  which 
Incorporates  sound  statistical  advice  directed  towards  obtaining  as  much 
Information  as  possible  from  firing  performance  data. 

D.  XM1  Trials  Below  *2B,F.  This  portion  of  the  briefing  quickly  des¬ 
cribed  the  second  test  matrix,  a  matrix  with  slightly  lower  priority  .than 
the  first.  Table  5  gives  the  firing  lists  with  the  numbering  of  Table  3. 

TABLE  5:  Prioritized  Lists  of  Trials  for  Each  Crew,  XMl  Alone, 
Below  "2B°F  (Numbering  from  Table  3) 


Crew  1 

Crew  2 

Crew  3 

5,12,14,19 

1,  8,16,21 

4,  9,13,20 

2,  7,17,24 

6, 11  ,'15, 22 

msji _ 

2,  7,17,24 

4,  9,13,20 

1,  8,16,21 
5,12,14,19 
3,10,18,23 

_ UUL&  .  ■ 

6.11.13.20 
2,  7,15,22 
3,10,14,19 
1,  8,18,23 

5.12.16.21 

_ ihlalliH 

The  lists  In  Table  S  are  very  similar  to  those  In  Table  4.  In  fact  the  only 
difference  In  the  two  sets  of  lists  Is  that  In  the  lists  of  Teble  5: 

•  Each  crew  fires  Its  HEAT  rounds  In  the  opposite  order  from  that  of 
Table  4. 

•  The  APDS  and  APFSOS  rounds  fired  In  HEAT  versus  KE  trials  are  those 
fired  In  APDS  versus  APFSDS  trials  In  Table  4. 

E.  XMl  versus  M60  Trials,  This  portion  of  the  briefing  Introduced  the 
new  problem  with  comparison  firing  and  described  the  proposed  solution, 
emphasizing  the  similarity  of  XMl  versus  MOO  trials  to  those  with  XMl  alone. 
Comparison  trials  were  to  have  substantially  lower  priority  than  trials  for 
XMl  alone. 
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Figure  2.  Cumulative  Completion  of  Celle  In  the  Boeto  Matrix. 

(Crew  Numbers  of  Crewe  Firing  Eooh  Cell  Are  Shown.) 
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The  new  problem  was  that  for  comparison  firing,  two  tanks  and  two  craws 
must  ba  prasant  for  every  trial.  Tha  proposal  was  that  aach  craw  flra  four 
missions  during  a  trial,  two  from  aach  tank,  so  that  a  trial  would  consist 
of  aloht  missions  rathar  than  four  missions  as  In  XMl-alone  trials.  Each 
craw  would  flra  at  both  rangas  and  at  both  stationary  and  moving  targats, 
but  ona  craw  would  flra  only  from  a  moving  tank,  and  tha  othar  craw  would 
flra  only  from  a  stationary  tank.  Each  craw  would  flra  only  ona  round  typa, 
but  aach  craw  would  flra  tha  sama  combinations  of  tank/targat  mods,  ranga, 
and  round  typa  for  both  tanks.  In  a  typical  trial,  tha  craws  could  ba  craw 
1  and  craw  2,  say,  and  axacuta  tha  following  missions  during  ona  firing 
parlod: 


Craw  1 

XM1,  S/S,  HEAT,  1500 
XM1,  S/M,  HEAT,  2500 
M60,  S/S,  HEAT,  1500 
M60,  S/M,  HEAT,  2500 


Craw  2 

XM1,  M/S,  ARDS,  2500 
XM1,  M/M,  AP0S,  1500 
M60,  M/S,  APDS,  2500 
M60,  M/M,  APDS,  1500 


Thasa  trials  ara  actually  vary  similar  to  thosa  for  XM1  alona.  In  fact,  tha 
pattern  for  aach  tank  Is  exactly  tha  pattern  for  tha  specific  example  of  a 
typical  trial  for  XM1  alone  above  -2S°F: 

S/S,  HEAT,  1500 
S/M,  HEAT,  2500 
M/S,  APDS,  2500 
M/M,  APDS,  1500 

(This  ongoing  similarity  should  ba  comforting  to  tha  test  operator;  It 
shows  that  tha  proposed  design  presents  essentially  one  obstacle  to  control 
of  trials,  not  many.) 

The  order  of  mission  conduct  within  each  trial  should  bo  randomized  as 
much  as  possible  within  test  constraints,  but  only  limited  randomization 
would  probably  be  possible.  In  the  typical  trial,  craw  1  might  ba  In  tha 
XM1  on  a  mobility  run,  and  crew  2  would  be  due  to  replace  crow  1  In  tha  XM1. 
It  would  ba  sensible  for  tha  test  operator  to  have  craw  2  flra  Its  missions 
In  tha  M60  before  tha  XM1  arrives,  than  have  craw  1  flra  Its  XM1  missions, 
change  craws,  have  craw  2  flra  Its  missions  (which  frees  tha  XM1  for  another 
mobility  run),  and  finally  have  craw  1  flra  Its  M60  missions. 

Detailed  scheduling  lists  are  given  In  Table  6.  As  with  XMl-alone 
trails,  they  should  be  conducted  top/down  If  all  goes  wall,  and  bottom/up  If 
problems  arise.  Tha  ordering  and  Its  benefits  Is  also  similar  to  XMl-alone 
trials.  The  obvious  problem  with  these  lists,  however,  Is  that  they  require 
specific  pairs  of  craws  to  ba  prasant  for  aach  trial,  which  Isnoses  awkward 
scheduling  difficulties  on  tha  test  operator.  These  difficulties  ara  un¬ 
avoidable  since,  for  Instance,  If  crew  1  and  craw  2  ware  to  flra  eight 
trials  together  rathar  than  the  planned  six,  then  no  craw  would  be  available 
as  a  partner  for  craw  3  on  two  trials.  For  all  craws  to  flra  the  same  num¬ 
ber  of  trials  when  two  craws  are  necessary  for  each  trial,  aach  pair  of 
craws  must  flra  the  same  number  of  trials  together. 
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TABLE  6:  Prioritized  Lists  of  Trials  for  Each  Pair  of  Crews, 
XM1  versus  M60,  Above  -25°F  (Numbering  from  Table  3) 


Crew  1/Crew  2 
(XM1;  M6Q/XM1 ;  M60^ 

4,  9;  4,  9/13 ,20; 13, 20 
17, 24; 17, 24/  2,  7;  2,  7 
14,19;14,19/  5,12;  5,12 
1,  8;  1,  8/16, 21; 16, 21* 
15,22;15,22/  6,11;  6,11 
3.10;  3.10/18.23:18.23 

*Th1s  Is  the  specific  ex 


Crew  1/Crew  2 
CXMli  M6Q/XM1:  M60) 

2,  7;  2,  7/17, 24; 17, 24 
13, 20 ;13,20/  4,  9;  4,  9 
16 , 21 ; 16 , 21/  1,  8;  1,  8 
5,12;  5, 12/14 ,19; 14, 19 
18,23;18,23/  3,10;  3,10 
-1,11; , 6,11/15, 22jl5, 22 

iple  of  a  typical  trial. 


Crew  1/Crew  2 
(XM1:  M60/XM1:  M60) 

14, 19; 14, 19/  5,12;  5,12 
1,  8;  1,  8/16,21;16,21 
4,  9;  4,  9/13, 20; 13, 20 
17 ,24 ;17 , 24/  2,  7;  2,  7 
3,10;  3,10/18,23;18,23 
15 , 22 ; 15 , 22/  6,11;_6,11 


F.  XM1  Side  Tests  and  Make-up.  This  portion  of  the  briefing  described 
how  any  rounds  left  over  from  the  main  design  could  be  used. 


In  the  unlikely  event  that  the  main  design  could  be  conducted  quickly 
without  major  deviations  from  the  plan,  the  remaining  rounds  could  be  fired 
In  eight  trials,  each  trial  fired  by  one  crew  during  one  firing  period  and 
consisting  of  seven  missions  (five-round  shot  groups)  from  a  stationary  tank 
against  stationary  targets: 

HEP  at  1500  meters, 

HEP  at  2000  meters, 

APDS  at  1500  maters, 

APDS  at  2000  meters, 

APDS  at  2500  meters, 

APDS  at  3000  meters, 

APFSDS  at  either  2000  meters  or  3000  meters  but  not  both. 


Trials  should  be  balanced  over  crews  and  temperatures  as  much  as  possible, 
and  half  of  the  APFSDS  missions  should  be  fired  at  each  of  2000  meter  and 
3000  meter  ranges.  Order  of  missions  within  each  trial  should  be  randomized 
as  much  as  test  conditions  permit. 

If  conducted,  these  trials  could  provide  some  Insight  to  HEP  perform¬ 
ance  In  the  cold  and  to  KE  performance  at  ranges  not  addressed  In  the  main 
design.  In  the  more  likely  event  that  during  the  conduct  of  the  main  design 
extra  rounds  were  needed  for  zeroing,  diagnostic  testing,  or  re-executing 
partially  completed  trials,  this  last  matrix  provides  a  store  of  low  prior¬ 
ity  rounds  for  use. 

III.  TECHNICAL  ASPECTS  OF  THE  DESIGN.  The  previous  section  of  this 
paper  described  the  proposed  design  "and  Its  advantages  In  nontechnical 
terms.  This  section  discusses  briefly  how  the  design  was  constructed  and 
sketches  Its  analytic  properties. 


The  key  to  the  design  Is  Its  blocking  scheme,  which  Is  based  on  P.  W. 
M.  John's  three-quarter  replicates  (1).  Ignoring  for  the  moment  the  two  KE 
round  types,  there  are  four  primary  factors  of  Interest,  each  at  two  levels: 

Factor  A,  tank  mode  (stationary,  moving), 

Factor  B,  target  mode  (stationary,  moving), 

Factor  C,  range  (1S00  meters,  2500  meters), 

Factor  D,  round  type  (HEAT,  KE). 

For  firing  of  the  XM1  alone,  at  each  temperature  range,  each  crew  was  to 
execute  the  resulting  24  design  In  blocks  (trials)  of  four  missions  with 
defining  contrasts: 

(1)  I  ■  +AD  ■  +ABC  ■  +BCD, 

(11)  I  ■  -AD  ■  ♦ABC  ■  -BCD, 

(111)  I  «  +AD  -  -ABC  ■  -BCD, 

CIV)  I  *  -AD  “  -ABC  ■  +BCD. 

With  this  blocking  schema,  all  main  affects  and  all  2-factor  Interactions 
(except  AD)  can  be  estimated  free  from  blocks  and  other  2- factor  Interac¬ 
tions.  Moreover,  If  one  of  the  blocks  Is  missing,  the  remaining  12-point 
design  Is  a  saturated  fraction  with  all  main  effects  and  2-factor  Interac¬ 
tions  (except  AD)  still  estimable  free  from  blocks.  (After  thinking  about 
tank  gunnery,  It  was  felt  that  AD  and  BD  were  likely  to  be  the  least  Impor¬ 
tant  2-factor  Interactions). 

The  problem  of  two  KE  round  types  was  solved  by  letting  KE  round  be  one 
type  In  the  blocks  having  +BCD  In  their  defining  contrast  and  the  other  type 
In  blocks  having  -BCD  In  their  defining  contrast,  and  running  two  more 
blocks  with  only  KE  rounds.  Using  the  coding  from  Table  4,  this  yielded  two 
possible  blocking  schemes  for  conducting  the  24-point  design  In  6  blocks: 


Block 

Scheme  1 

Scheme  2 

1 

2,  7,15,22 

2,  7,17,24 

2 

6,11,13,20 

4,  9,13,20 

3 

1,  8,18,23 

1,  8,16,21 

4 

3,10,14,19 

5,12,14,19 

5 

4,  9,17,24 

3,10,18,23 

6 

5,12,16,20 

6,11,15,22 

Ignoring  KE  round  type,  blocks  1-4  correspond  to  the  defining  contrasts 
(1)-(1v)  In  the  previous  paragraph.  An  examination  of  Tables  4  and  5  shows 
that  Scheme  1  was  used  to  construct  the  lists  for  crews  1  and  2  above  -25°F 
and  for  crew  3  below  -25°F,  while  Scheme  2  was  used  to  construct  the  lists 
for  crew  3  above  -25°F  and  for  crews  1  and  2  below  -25°F.  The  Introduction 
of  two  KE  round  types  In  this  manner  gives  the  design  an  Incomplete  blocks 
aspect:  the  effect  of  round  type  Is  partially  confounded  with  the  BC  Inter¬ 
action,  and  additional  Information  about  the  effect  of  round  type  can  be 
obtained  from  an  Interblock  analysis.  All  effects  of  Interest  can  still  be 
estimated  with  any  one  block  missing  or  one  of  the  HEAT-versus-KE  blocks  and 
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an  appropriate  KE-only  block  missing.  (Since  any  difference  between  KE 
round  types  measurable  from  this  experiment  was  likely  to  be  negligible,  the 
analysis  in  practice  was  likely  to  proceed  as  if  there  were  only  one  KE 
round  type. ) 

For  XMl-alone  trials  in  either  temperature  range,  the  planned  trial 
order  was  chosen  so  that  the  first  four  trials  for  any  crew  were 
KE-versus-HEAT  trials  and  so  that  the  Nth  trials  on  all  three  crews  repre¬ 
sented  three  of  the  four  different  blocks  with  the  same  two  round  types. 
Thus  at  either  temperature  range,  any  three  of  the  first  four  trials  on  any 
crew  constituted  a  three-quarter  replicate  if  KE  round  type  were  ignored, 
and  the  Nth  trial  on  all  three  crews  constituted  a  three-quarter  replicate 
confounding  crew  with  blocks. 

Taking  both  XHl-alone  firing  matrices  together,  the  design  Is  a  split- 
plot  design.  The  subplots  are  missions  (shot  groups)  treated  by  tank/target 
mode,  range,  and  round  type  using  a  factorial  scheme.  Even  with  substantial 
data  loss,  clear  inference  concerning  subplot  factors  should  be  possible 
since  presence  of  even  one  three-quarter  replicate  guarantees  estl inability 
of  interesting  effects.  The  main  plots  are  trials  (blocks)  treated  by  round 
combination,  crew,  order,  temperature  range,  temperature,  and  additional 
random  error.  With  some  luck,  inference  concerning  main  plot  factors  should 
be  possible.  In  the  unlikely  event  that  the  entire  XMl-alone  design  could 
be  run  as  planned,  quite  elaborate  analyses  would  be  possible,  one  of  which 
is  Indicated  In  Table  7.  With  only  moderate  data  loss,  analysis  along  the 
lines  of  that  In  Table  7  could  probably  still  be  conducted  with  some  suc¬ 
cess.  As  in  all  split-plot  designs,  however,  care  must  be  taken  with  the 
error  terms. 

The  designs  corresponding  to  the  remaining  two  test  matrices  were  not 
as  neatly  structured  as  the  design  corresponding  to  the  XMl-alone  matrices. 
The  lowest  priority  XM1  firing  subtest  was  essentially  a  nonstatistlcal 
demonstration  subject  to  cannibalization  for  rounds.  The  XM1  versus  M60 
comparison  had  lower  priority  than  XMl-alone  testing,  and  by  emphasizing  the 
comparison  between  tanks,  the  design  lost  much  of  Its  analytic  potential 
concerning  other  effects.  For  any  particular  pair  of  crews  and  either  tank 
type,  the  same  blocking  scheme  used  previously  was  exploited  by  confounding 
crew  effect  with  the  ABCD  Interaction,  which  confounds  (crew)x(tank  mode) 
and  (crew)x(round  type)  with  blocks.  Three-quarter  replicates  were  still 
preserved,  but  with  lower  resolution  for  A,  B,  C,  and  D  (main  effects  only). 
By  crossing  tank  type  with  the  design  in  the  other  factors,  however,  maximum 
Information  about  tank  effects  was  obtained  and  potential  for  simple  and 
easily  presented  paired-comparison  analysis  was  Introduced. 
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TABLE  7:  Possible  Analysis  of  Variance  for  Trials  Involving  XMi  Alone. 
Assuming  All  Trials  Run  Successfully 


Source 

DF 

Source 

OF 

1. 

Total  Sum  of  Squares 

143 

2. 

Blocks  (Whole  Plots) 

35 

3. 

Treatments  (Subplots) 

19 

2.1 

Round  Combination 

2 

3.1 

A  (Tank  Mode) 

1 

2.2 

Crew 

2 

3.2 

B  (Target  Node) 

1 

2.3 

Order 

1 

3.3 

A  x  B 

1 

2.4 

Crew  x  Order 

2 

3.4 

C  (Range) 

1 

2.5 

Temperature  Range 

1 

3.5 

A  x  C 

1 

2.6 

Temperature 

1 

3.6 

B  x  C 

1 

2.7 

(Temperature)8 

1 

3.7 

D  (Round  Type) 

2 

2.8 

Whole  Plot  Error 

3.B 

C  x  D 

2 

[2-(2. !♦. . .+2.7)] 

25 

3.9 

A  x  Crew 

2 

3.10 

B  x  Crew 

2 

3.11 

C  x  Crew 

2 

3.12 

A  x  Temperature  Range 

1 

3.13 

B  x  Temperature  Range 

1 

3.14 

C  x  Temperature  Range 

1 

Subplot  Error 

89 

IV.  SUMMARY  AND  CONCLUSION.  The  goals  for  cold  regions  testing  of  XMI 
firing  performance  were  ambitious,  and  a  large  number  of  rounds  were  avail¬ 
able  for  test.  I  can  Justify  neither  a  statistically  naive  nor  a  statis¬ 
tically  pure  approach  to  such  testing.  By  working  from  basic  statistical 
principles  tempered  by  a  concern  for  operational  constraints,  I  was  able  to 
devise  what  I  believe  was  a  statistically  sound  and  operationally  executable 
design  for  this  particular  test.  By  suppressing  some  statistical  niceties 
and  most  technical  jargon,  I  was  able  to  sell  the  design  to  the  test  opera¬ 
tor  In  the  sense  that  he  agreed  to  attempt  It  along  the  proposed  lines. 
Once  this  agreement  was  reached,  I  was  able  to  gain  control  of  certain 
detailed  planning  tasks  through  which  I  made  actual  execution  of  the  pro¬ 
posed  design  more  likely.  The  design  proved  flexible  In  that  modifications 
could  be  made  easily  as  test  planning  progressed.  In  particular,  concern 
with  comparison  of  two  KE  round  types  was  eventually  dropped  (with  the 
obvious  design  modification),  and  an  eventual  reduction  to  two  crews  was 
easy  to  accommodate.  Unfortunately,  temperatures  during  the  test  season 
were  exceptionally  warm,  and  when  a  few  days  of  appropriately  cold  condi¬ 
tions  finally  arrived,  tank  malfunctions  precluded  firing  performance  test¬ 
ing.  No  rounds  were  fired  for  record.  Nevertheless,  I  believe  this  paper 
shows  that  sophisticated  designs  for  field  tests  are  not  only  feasible  but 
also  marketable  to  the  testing  community  If  technical  scruples  are  not 
allowed  to  dominate  potential  bottom-line  results. 
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LONG-TERM  STORAGE  OF  ARMY  RATIONS 


Edward  W.  Rota,  Jr. 
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ABSTRACT .  An  Important  part  of  tha  Army's  efforts  to  Improve  the 
food  available  to  soldiers  Is  a  program  of  testing  the  ability  of  rations 
to  maintain  their  acceptability  when  stored.  This  paper  describes  an 
experiment  now  underway  on  one  of  tho  Army  combat  rations,  the  Meal, 
Ready-To-Eat  (MRE).  In  this  paper  we  are  mainly  concerned  with  the 
statistical  methods  used  to  analyse  data  of  the  type  obtained  In  this 
experiment  in  order  to  estimate  the  shelf-life  of  the  food  involved. 

I.  INTRODUCTION.  This  report  Is  about  the  treatment  of  data 
from  a  large-scale,  long-term  storage  experiment  on  a  certain  type  of 
Army  ration,  called  the  Meal,  Ready-To-Eat  (MRE).  The  main  purpose  of 
the  test  is  to  find  the  effect  of  storage  at  various  temperatures  on 
the  acceptance  of  this  ration,  i.e.,  how  well  it  is  liked  by  its 
consumers.  There  are  a  number  of  interesting  questions  Involved  in 
gathering  and  analysing  this  data  as  well  as  interpreting  the  results 
to  potential  users. 

II.  THE  EXPERIMENT  AND  THE  DATA.  The  experiment  consists  of  pur¬ 
chasing  the  rations,  testing  a  sample  of  each,  then  storing  the  remain¬ 
der  at  four  different  temperatures,  4,  21,  30  and  38°C.  After  specified 
time  intervals,  more  samples  are  withdrawn  from  storage  and  tested, 

and  the  results  compared  with  those  obtained  earlier.  The  schedule  of 
storage  times  and  temperatures  is  shown  in  Table  1. 

The  rations  consist  of  twelve  menus,  each  comprising  four  or  five 
items.  The  total  number  of  items  is  52,  provided  the  same  food  in  two 
different  menus  is  viewed  as  two  different  items.  When  the  ration  is 
tested,  all  the  items  in  it  are  presented  to  each  of  36  people.  Each 
person  rates  each  item  in  the  menu  on  a  scale  of  1  through  9,  where 

9  means  "like  extremely" 

* 

5  means  "neither  like  nor  dislike" 

0 

* 

1  means  "dislike  extremely." 

Thus,  for  each  of  the  29  combinations  of  storage  duration  and  tempera¬ 
ture,  there  are  52  x  36  -  1872  scores  to  be  analysed. 

There  are  a  number  of  easily  perceptible  difficulties  with  this 
test  plans 

(a)  The  amount  of  data  that  will  eventually  be  accumulated  may 
be  quite  large. 
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(b)  Th«  pool  of  test  subjects  i*  essentially  the  work  forc«  at 
Natick  Laboratorias,  which  parhapa  differs  somewhat  in  composition 
from  the  consumer  population  for  whom  the  meal  is  intended)  i.e.  the 
Armed  Forces. 

(c)  The  long  duration  of  the  test  may  cause  various  kinds  of 
trouble.  E.g.,  the  tastes  of  the  test  pool  or  population  may  drift, 
and  there  may  be  changes  in  the  people  giving  the  test  and  analysing 
the  data,  caused  by  death,  retirement  or  job  actions. 

<d)  The  form  of  tho  date  is  a  source  of  some  uncertainty  as  to 
the  appropriate  method  of  analysis.  Most  statistical  data  is  oithor 
continuous  or  categorical  with  a  few  categories  (typically  £4). 

Our  data  are  ordinal  and  categorical  with  nine  categories,  which  places 
it  in  an  intermediate  situation  where  neither  kind  of  analysis  is 
wholly  satisfactory. 

Some  of  these  issues  will  be  discussed  in  tho  subsequent  sections. 

111.  ANALYSIS  OF  DATA.  For  each  food  at  each  combination  of 
time  and  temperature  the  data  form  a  histogram  for  the  frequency  of 
each  integer  in  the  rango  1  through  9,  the  total  count  being  36. 

From  this  information  we  wish  to  characterize  the  acceptability  of  the 
food  when  stored  for  various  times  and  temperatures. 

Tharo  are  many  ways  of  characterising  the  storage  stability  of  a 
food  with  regard  to  consumer  acceptance.  A  common  ingredient  in  most 
such  descriptions  is  tho  definition  of  a  critical  scoro,  with  tho 
property  that  the  food  is  pronounced  unacceptable  if  its  score  falls 
bulow  the  critical  scoro.  Usually  the  critical  scoro  is  taken  as  5. 

Given  this,  we  shall  regard  as  basic  the  idea  of  shelf-life,  , 
at  storage  temp^raturo  H.  is  the  time  in  months  required  for 

food  stored  at  temporuture  11  to  obtain  a  scoro  of  3.  Wo  assume  that 
the  initial  scoro  >  3,  for  otherwise  the  food  would  not  be  in  the 
system. 

Other  parameters  which  characterize  storage  stability  are,  e.g., 
the  average  score  after  a  fixod  storage  period  (say  12  months).  This 
is  less  useful  than  but  ussier  to  ostimato.  If  more  were  known 

about  the  relationship  among  score,  tempersture  and  storage  time,  it 
might  bo  possible  to  define  a  single  parameter  which  would  predict 
all  combinations  of  time  and  temperature  that  cause  a  score  of  3  for 
u  food.  The  prosont  data  may  lead  to  such  a  description,  but  wo  shall 
not  pursue  it  further  here. 

We  have  already  mentioned  the  fact  that  neither  a  categorical 
treatment  (i.e.  via  contingency  tables)  nor  a  continuous  approach 
(e.g.,  linear  regression)  is  wholly  satisfactory  in  analysing  this 
duta.  Tho  categorical  treatment  does  not  load  easily  to  a  prediction 
of  shelf  life,  still  lass  to  estimates  of  its  variance.  The  continuous 
methods  assume  a  Guussian  distribution  of  scores,  which  is  not  satisfied. 


The  scheme  we  adopt  uses  both  methods  in  an  attempt  to  avoid  the 
pit£alle  faced  by  each  separately •  Moreover,  it  carries  out  the 
analysis  at  two  levels  of  intensity  on  the  data  for  each  food  and 
storage  temperature  up  to  the  current  time.  First  a  coarse  computation 
is  done  to  determine  whether  the  scores  have  changed  during  the  time 
of  the  test.  If  they  have  not,  we  record  the  histogram  of  scores  up 
to  the  current  time,  calculate  the  mean  and  standard  deviation  of 
those  scores,  but  do  no  computation  of  shelf  life  (which  is  effectively 
Infinite  in  this  eventuality).  If  tha  coarse  computation  shows  signi¬ 
ficant  change  in  scores,  we  do  two,  more  elaborate  analyses  in  order 
to  predict  shelf  life.  We  describe  both  the  coarse  and  elaborate 
analyses  in  tho  following  paragraphs. 

The  coarse  analysis  uses  two  methods,  a  contingency  table  analysis 
and  a  linaar  laast  squaras  calculation.  The  contingency  table  analysis 
is  done  twice,  once  with  ell  non-empty  columns  end  then  with  only  2 
columns,  usually  obtained  by  pooling  scores  1  through  6  and  7  through 
9,  Tha  tall  probabilities  associated  with  the  chi-square  tost  are 
recorded  for  both.  Also,  tha  tall  probability  associated  with  tho 
F-tast  of  the  hypothesis  that  tha  slope  is  stro  is  recorded  from  the 
regression.  If  any  of  these  three  tell  probabilities  is  smell  enough 
(usually  <.10  or  oven <  .20),  tho  more  elaborate  analysis  is  done. 

In  addition,  estimates  of  ths  shelf-lift  and  a  90%  lower  confidence 
limit  art  recorded  if  tha  ragrasaion  slope  is  non-sero. 

Tho  elaborate  analyses  apply  a  non-linaar  least  squares  (NULS) 
and  a  multinomial  logit  method  to  the  data.  Tha  non-linaar  laast 
squares  procedure  is  basod  on  the  model 


X1  + 


t  <  x. 


(is.  -  5)(t  -  x.)  > 


XT  ■  [x1#x2»x3] 


whore  y  is  the  score,  t  the  storage  duration  end  X  is  the  parameter 
vector  fitted  by  tha  least-squares  process.  Tho  X~compon«nts  have  tho 
meanings 


-  initial  score 
Xj  •  shelf  llfo  (months) 
x^  ■  lag  or  Induction  period  (months) 
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see  Figure  1.  e  is  sn  i.i.d  Gaussian  (0,S2  )  random  variable.  The 
non-linear  least-squares  program  NL2S0L  was  used  to  estimate  X  and 
its  Hessian,  from  which  confidence  limits  were  obtained. 

The  multinomial  logit  method  estimates  the  histogram  of  score 
probabilities 

9T(t)  -  (6i(tM2<fc),">»6q(t)] 


where  9j(0  Is  the  probability  of  score  j  at  time  t  .  We 

assume  the3 logit  model 

0j(t)  -  expC-YjftJvJljaxpC  -Yk(t» 

< 

V*>  ■  AV«U<0 

♦lj  '  3  -  5  1  *2j  “  *1J 

*3j  ■  ♦xj**/12  *  $4j  ■  ♦jj**/12 

♦5j  "  ’  10>/2°  •  *6J  •  ♦Sj-1'12  • 


and  U  is  the  vector  of  parameters  to  be  fitted, 
T 

U  -  . 


The  estimation  of  U  Is  done  by  minimising  the  negative  logarithm 
of  tho  likelihood  of  getting  the  observed  counts  ,  r  (t),  J  ■  l»*‘*»q 
which  leads  to  minimising  J 

"t  1  q 


Nt  is  the  number  of  times  at  which  we  have  data.  Having  solved  for 
U*  the  shelf  life  is  found  by  solving 
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<Kt,u) 


•  o 


-  JS1<J  -  5)exp(~Yj (t)) 


for  t.  A  linearised  estimate  of  the  variance  in  ahalf  lift  la  alto 
obtained  from  related  formulae.  A  ipecial  purpose  minimiter  of  Newton 
type  wat  written  to  tolvc  for  U  and  the  XMSL  veralon  of  the  Brent 
algorithm  furnlahed  the  shelf-life  ectimate. 

Theta  elaborate  analyaea  require  eontlderable  computation  but 
provide  a  good  deal  of  information  about  the  food.  In  particular,  we 
obtain  eatimatet  of  thelf  life  and  90%  lower  confidence  llmlte  from 
both  methoda.  Alto,  each  method  allowt  ut  to  predict  the  average  tcore 
for  any  tlmo,  and  the  logit  method  predictt  the  complete  hiatogram  for 
any  time.  Obviously,  prediction  too  far  into  the  future  by  either 
method  it  rlaky. 

Both  methoda  require  non-linear  minimlcation  to  aolve  for  their 
unknown  parameter!  and  can,  therefore,  encounter  a  variety  of  diffi- 
cultiea,  e.g.,  non-convergence,  convergence  to  a  local  but  not  global 
minimum,  alngularicy  of  the  Haitian,  etc,  Both  minlmisara  contain 
aome  guard*  agalnat  theae  peril*,  and  an  additional  check  it  furnlahed 
by  comparing  the  reaultlng  ahelf-livea,  but  abaolute  certainty  la 
not  poaalble.  The  two  method*  need  not  lead  to  the  aame  (half-life 
eatlmate*,  though  we  expect  them  to  be  reasonably  cloae  if  both  con¬ 
verge  wall. 

Moat  of  the  computet  Iona  were  done  by  meant  of  the  1MSL  sub- 
routlnea  for  forming  and  analyalng  contingency  tablet  and  doing  leaat- 
aquarea.  The  LINPACK  aubroutlnea  were  uaed  extenalvely  in  the  mlnlmiser 
for  the  logit  method. 

IV.  INTERPRETATION.  An  intereating  aapect  of  the  preaent  pro¬ 
blem  concarna  the  reporting  of  reaulta,  l.e.,  how  much  of  what  kind 
of  Information  should  be  relayed  to  the  food  technologiata  and  thence 
to  the  loglatical  planner*  and  purchasing  agents.  It  la  clear  that  a 
lot  of  information  la  produced  at  each  atago  by  the  computations, 
some  of  which  is  not  directly  useful  to  the  food  technologiat. 

At  the  current  time,  after  12  months  of  storage,  the  Information 
roportod  to  the  technologiat  la  shown  in  Table  II  and  Figure  2.  The 
first  is  the  more  Important.  It  is  a  table  of  foods  and  storage 
temperatures  whose  gentries  are  the  ahelf-livea  of  foods  estimated  to 
have  ahelf-livea  ■  24  months.  Estimates  are  listed  only  if  they 
have  some  credibility,  l.e.,  in  this  case  a  907,  lower  confidence  limit 
which  is  positive.  Foods  with  shelf  lives  >  24  months  are  currently 
estimated  with  poor  accuracy  alnce  all  the  data  is  for  •  12  months. 

Food  technologists  are  occasionally  asked  about  the  mean  scores 
of  various  foods  during  the  test.  Figure  2  gives  the  mean  food  score 
at  the  moat  recent  time  of  test,  l.e.,  12  months,  in  the  form  of  a 
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histogram.  Each  food  li  represented  by  a  3-character  plotting  symbol, 
of  which  the  first  two  characters  are  the  food  number,  and  the  third 
is  either  S  or  D,  S  if  the  food  score  was  Judged  to  be  stable  and  D  if 
deteriorating.  For  foods  labelled  S  the  mean  score  was  calculated 
over  all  times  up  to  the  current  test,  but  for  0  the  mean  is  taken 
for  the  most  recent  time  of  test  only. 

For  example.  Table  2  shows  that  food  number  2,  ham-chicken  loaf, 
had  a  shelf-life  estimated  as  13  months  at  38°  storage  temperature. 

Xn  Figure  2  the  histogram  for  38°  shows  that  food  number  02  had  a  mean 
score  of  3.1  and  was  deteriorating  at  12  months. 

For  the  sake  of  ready  reference,  a  simple  table  of  mean  scores 
of  all  foods  at  the  four  storage  temperatures  was  also  given  to  the 
technologists.  However,  the  histogram  is  in  most  respects  a  more 
useful  form  for  this  information. 

V.  EXAMPLE.  In  FigureaS  thru  9  we  show  examples  of  input  and 
output  pr'oduc'eci  by  the  programs  in  the  course  of  analysing  the  data 
for  food  number  34,  fruit  mix,  after  updating  with  the  new  scores  at 
12  months. 

Figure  3  shows  the  input,  consisting  of  ton  lines  of  data.  At 
this  time  the  data  comprises  two  lines  for  4°  and  21°  (at  0  and  12 
months)  and  three  lines  for  30°  and  38°  (at  0,  6  and  12  months). 

Figures  4  and  3  present  the  output  from  the  coarse  analysis. 

For  4°  and  21°  only  two  times  are  available  so  ths  contingency  tables 
consist  of  only  two  rows  plus  a  sum  row,  The  program  does  a  t-test 
for  a  difference  in  means  Instead  of  the  linear  regression.  At  30°  and 
38°  there  are  three  times,  hence  3  rows  in  the  contingency  table,  and 
the  linear  regression  is  done.  The  last  line  writes  the  smaller  of 
the  two  tall  probabilities  from  the  contingency  table  and  the  tall 
probability  from  the  t-test  or  linear  regression. 

We  see  that  for  temperatures  4,  21  and  30  degrees,  none  of  the 
tall  probabilities  were  lass  than  .10,  the  critical  value  used  here. 

At  38  degrees,  however,  both  contingency  table  methods  and  the  linear 
least  squares  analysis  gave  probabilities  <  .10,  i.s.,  both  methods 
agreed  that  the  food  scores  had  changed.  For  many  foods  and  tempera¬ 
tures  the  results  were  not  as  clear  as  in  this  example. 

Figures  6-9  show  the  results  of  the  more  elaborate  analysis, 
done  only  for  the  38°  storage  case.  Figure  6  Is  the  printout  from 
the  NLLS  analysis.  We  see  the  process  converged  after  6  iterations 
because  C0NV«4,  i.e.,  the  gradient  became  small,  and  the  solutlona  were 

X}  ■  initial  score  *  7.19 

X2  ■  Xjg  *  20.7  months 

X3  ■  lag  or  Induction  period  =>3.4  months, 

the  lower  90%  confidence  limit  for  the  shelf  life  being  16.3  months. 
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The  printout  and  plot  of  tha  data  and  the  flttad  function  show  that 
at  the  next  withdrawal  (18  months)  the  average  score  la  predicted  to  be 
about  5.39.  This  plot  shows  the  upper  and  lower  quartilea  of  the  data 
at  each  storage  time  as  a  "U"  and  "L",  respectively.  The  mean  at  each 
time  is  plotted  at  "0",  but  an  “MM  is  printed  wherever  two  plotted  points 
coincide,  as  the  theoretical  and  data  means  do  here. 

The  logit  results  depicted  in  Figures  8  and  9,  give  information  first 
about  the  minimisation  process ,  including  estimates  of  the  parameters 
U}  to  U5  and  comparisons  of  the  experimental  and  predicted  score  counts. 

The  results  of  the  shelf-life  calculation  are  then  stated,  followed  by 
the  predicted  counts  and  mean  score  at  the  next  withdrawal  period  (18 
months).  We  see  that  the  shelf-lifs  is  estimated  as  20.3  months,  which 
agrees  quite  well  with  the  estimate  of  the  NLLS  calculation.  The  lower 
confidence  limit,  10.8  months  is  appreciably  lower  than  the  16.5  months 
given  by  NLLS  method.  The  model  predicts  at  18  months  a  mean  score  3.42, 
not  much  higher  than  the  3.39  value  obtained  from  NLLS. 

The  comparison  between  experimental  and  theoretical  histograms  is 
shown  in  Figure  9,  where  'V  signified  (experimental  counts)  > 

(theoretical  counts)  and  the  reverse.  It  is  clear  that  tha  model 
does  not  reproduce  the  details  of  the  experimental  counts  very  well  in 
this  case. 

Sometimes  the  difference  between  the  logit  and  NLLS  calculations 
was  greater  than  in  this  example  but  seldom  exceeded  4  months  when  both 
procedures  converged.  However,  there  were  many  cases  where  only  one  con¬ 
verged,  or  both  converged, but  one  or  both  had  poorly  conditioned  Hessian 
matrices.  Generally,  the  agreement  was  good  when  both  shelf-life  estimates 
were  <  20  months,  but  became  poorer  as  the  estimates  increased. 

VI,  CONCLUSIONS.  The  results  up  through  the  present  time,  12 
months  of  storage,  »Kow  that  only  two  of  the  52  foods  have  failed.  These 
are  frankfurters  (#6)  and  brownies  (#13).  Neither  result  is  certain. 
Frankfurters  had  scores  averaging  <  3  on  the  initial  test  and  have 
scored  >  3  on  subsequent  tests.  This  suggests  that  the  initial  lot  may 
have  been  uncharacteristically  bad,  but  it  also  calls  for  closer  scrutiny 
of  that  product.  Brownios  occur  in  two  menus,  as  foods  number  13  and  14. 

As  #13,  it  had  a  shelf-life  of  only  11  months  at  38°,  but  as  #14  it 
showed  no  change  at  all!  This  suggests  that  there  may  be  an  interaction 
between  this  food  and  some  of  the  other  foods  in  these  menus,  but  this 
too  requires  detailed  study. 

The  methods  used  in  this  statistical  treatment  are  apparently 
adequate  though  not  the  only  ones  possible.  E.g.,  a  different  approach, 
via  reliability  procedures,  is  possible,  and  tha  methods  based  on 
Information  theory  (see  Kullback  [1])  are  also  available.  The  pro¬ 
cedures  used  here  were  chosen  because  they  could  be  carried  out  with 
available  computer  programs  and  had  substantially  different  viewpoints 
toward  the  data. 
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Again*  tha  raportlng  toola  uaad  in  Saction  IV  appaar  Co  ba 
aatiafactory  to  tha  food  tochnologiaea  and  ara  not  too  tima-cbnauming 
to  axacuta. 

Cltarly,  tha  reaulta  will  changa  with  tha  paaaaga  of  cltna  and 
soma  ehangaa  in  mathod  may  baeomt  nacaaaary.  Zt  la  hoptd  that  tha 
pradictlva  capabilltlaa  of  thaaa  methoda  will  anabla  tha  food  each- 
nologiata  to  avoid  unnaeaaaary  tooting*  but  thia  ramaina  to  ba  aaan. 
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TAtlE 

storage  TEMP 
************ 

4  DEG  C 

21  DEG  C 

30  DEG  C 

38  DEG  C 


ll  TEST  PLAN 

STORAGE  DURATION  (MONTHS) 

ttttttttttttttttttttttttt 

t, 18,30,36,48, 60,168 
0,18, 18,84,30,36, 48, 60, 126 
0,6,18,18,84,30,36 
0,6,18,18i84,114 


317 


TABLE  23  ARE  SUMMARY  AT  12  MONTHS 

ttmmtmmtmmmtrmttf 


U  H 

ae  h 


3  S3 


MOlMN  I  •  « 
00  H  h  H  h  (VI 
1*1* 


«h  i  i  m 

*4  IU  (U  irt  VI  M  M  *1  (VI  H 


S'  'as 


a.  «* 

CfOH 


HO* 

1  Ui  M 
OH 

-a  •  •  •  • 

(U* 


I  I  CO  I  <*101  I  I  I  to  I  ft  I 

H  H  H  ■*  nj 


i  cu  lit  ro  i  i  i  i  i  i  i 


$ZH  Wh 

ipsfps!!*' 


UMAX 

UUM 

immK 


1 


In 


^s;Ns 


O  « 

^HtO  I  I  I  <»  t  t  I  I  I  I  I  I  I  I  I  I  I 


O  D  H 

Vi  sc  *  o»  r> 

JuiM  i  i  i  i  i  i  w  i  i  otnim  i  »  «*«  ai 

C  C  H  H  H  f*  H  H 

cSS-MU*>uiu»*«ni/i<»*4*4niruui'v40MU*4 
MJjj*  *  •*»*  •*  *4 


8S|||»h| 

ps^!p 

slaalesS 


am 


!*!g^2Sl 


u-28-"",u"B3S5#6*R«S«C5C 


!S“Ssl 

IkIsI 


U.«_l 


mi 


HliTOOMftS  Of  ACM  ICOftfl  At  1C  HOHTMt 


o  o  o 

0  LA 

-+i/>  m 

•  CB 

t\t  T  |/» 

—  m 

Q 

W  0  U)  U)  o 

a  ci 

wn 

«  ux  ru  tfx  P- 

r-  rv 

*■4 

^#trun 

ru  n 

LA 

la 

la 

LAOOlAOLAOQO 

n 

lo 

▼ 

fu»a^r«- 

«*  o  m 

m 

m 

(U 

ru-f 

^  OJ  ®  40 

o 

Q 

Q  O  O 

la  o 

«tf)QOQV>OQV> 

r- 

x# 

•o  •  ox  n  <,# 

-m  ox  ~  •+  mr-  •rucxj 

r>  ▼  m 

ru  ac 

o 

AJ 


o  o 

2  i 


<V  VMM 
ru  ai  xa  — 

Ol^UOOV) 

Q 

ru 

m  ru  •  in  v  ru 

•-* 

LA 

LA 

y» 

V)QQQI/IQ 

a  o 

LA 

o 

tn 

a 

ox 

ru 

-«  -*«•  c*  xn  m 

c*  ♦ 

ru 

« 

«• 

r- 

rx 

r> 

(vnnury^ 

^4  —4 

• 

cn 

S 

ru 

O  LA 

LA 

QOW 

(AOOOOLAUXLA 

AO 

LA 

o 

Q 

n  u> 

LA 

•  •*  CT 

ncu  v  c»p*NniftU)  a 

V 

▼ 

— • 

LO 

n  -* 

-* 

▼ 

i\j  vnf\jn#«  v 

«1  ▼ 

«"• 

m 

•v 

• 

3 

M 

Ml 

m 

u 

m 

Ul 

« 

m 

rm 

m 

M» 

M 

m 

Ml 

U 

m 

CM 

« 

— 

n 

m 

o  a 

*- 

m 

V  « 

& 

m 

mux 

UJ 

m 

V) 

LA  LA 

o 

M 

*■4 

V  A 

o 

M 

ru 

ru  ru 

« 

M 

Q  LA 

LA 

LA  LA  LA 

O 

(1  ** 

U»P 

ox 

la  ax  la 

r> 

* 

Ul  «• 

•4  H 

-v 

ru** 

w 

o  « 

LA  LA  LA 

VA  LA  LA  LA  LA  LA 

LA  O  O 

O  LA 

c 

w 

onu 

n  r>  nj  •  o*  o* 

A  ^  T 

•  ru 

3 

M 

n  n  f*i 

cxj  o  ru  ru  ru 

•  «• 

m  4» 

la 

M  W 

LA  LA  O 

O  LA  LA  O  LA  LA 

Q  LA  O 

O  LA  Q 

X 

« 

«<#• 

Nt»««4*4Ul 

•  m 

n  v*4 

o 

«• 

■v 

▼  r>  va  M 

—  f\) 

t  - 1 

o 

«t  M  «*  « 

»  •» 

•  M  M  M 

Ul 

M 

•K 

M 

* 

«• 

LA 

n  < 
cxi  r 


V  Q  Q  t 

»nuM 


** 

Vl 

O  LA 

LA 

M 

OX 

▼  LA 

V 

M 

© 

ruru 

*-• 

M 

C#  LA  LA 

a  a 

o 

M 

m  rn  r- 

«r  eo 

0» 

A 

r»  -*  ▼ 

AJ 

o  *• 

LA 

O  L/>  ‘A  O  *A 

ui  a 

now 

LA 

w  •» 

OX 

oo  r-  ^  n  o  «o 

UA  —4 

us  a*  ru 

CO 

O  XX 

rx 

M  *-«  AJ  AJ  LA  •« 

O  -4 

ru  CD  ® 

m 

M 

C# 

o 

ck/.cv.  v.dowi 

U.  LA 

LA  LA  Q  a  O 

a  a  la  o  o 

V  X* 

Cr 

—4 

«P'X/J-f-i>U'*r*'4 

AJ  CO 

•  UMM 

*r 

n 

«  M  4 

—4  (’» txj  rA  T  -4ni/i 

AJ  AJ 

•  MM* 

V  -4  »*»  <V  — 

*1  •  T  ^  © 

*  M  M  M  W  M  « 

»s. 


>  ol  o#  r-  to  la  mj^cwoor- u>i/s  ▼  rx  ru  *-*«»c*oor>-a>vn^  n"»ro~*«>c»«»r-coi/>,v  m  ru  «-*  • 
if-f,-r-r-r-i~r-f-r-r-u>',o‘.OLOti»u>LX»LOLOLoiALALAiALALAiAiALAM>''e  •cm-  -c  v  -c  'e  ▼  -r  -«r 


'320 


Best  Available  Copy 


F igure 


FOODNAME 
tttttttt 
FRUIT  MIX 
FRUIT  M IX 
FRUIT  MIX 
FRUIT  MIX 
FRUIT  MIX 
FRUIT  MIX 
FRUIT  MIX 
FRUIT  MIX 
FRUIT  MIX 
FRUIT  MIX 


T  H  N 
**  X*  XX 

0  4  34  78967744 69 7699S78973736877667379786? 

12  4  34  687638467578679787967647787765773778 

0  21  34  789677446976998789787868776678797867 
12  21  34  764868538848274877698768868978667777 
0  30  34  789677446976998789787868776678797867 
6  30  34  798976868788527856886778867885448378 
12  30  34  874848798786977786837777765286567776 
0  38  34  789677446976998789787868776678797867 
6  38  34  587587867478788478858788878787877788 
12  38  34  487447766637778888865434477787486958 


T  -  STORAGE  TIME,  MONTHS 
H  -  STORAGE  TEMP., DEG  C 
N  -  FOOD  ITEM  NUMBER 


Figure  3 
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Best  Available  Copy 
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ASSURING  QUALITY  THROUGH 
BALLISTIC  TESTING 


wU'.kwUm 


Michael  P.  McMlllar 
Product  Assurance  Directorate 
US  Army  Aviation  Research  and 
Development  Command 
St.  Louis,  MO 


ABSTRACT .  The  V..  taat  is  a  r>::-mdard  teat  used  to  meaaure  the  ballietic  toler¬ 
ance  of  armor  material.  This  test  Is  baaed  on  the  aesumptlon  that  the  armor 
material  lot  is  homogeneous.  This  paper  poses  the  question!  "If  the  lot  of 
armor  to  be  tasted  la  not  homogeneous  is  the  V50  test  still  appropriate?"  Also, 
how  should  the  Operating  Characteristics  (OC)  Curve  be  determined  given  the 
constraints  detailed  later  in  this  paper? 

I.  INTRODUCTION. 

In  lata  March  1981,  personnel  from  the  Army  Aviation  Research  and  Development 
Command  (AVRADCOM)  witnessed  a  ballistic  tolarance  test.  This  test  was  the  cul¬ 
mination  of  an  effort  to  substitute  test  coupons  in  lieu  of  helicopter  pilot  and 
copilot  crew  seats.  These  seats  are  required  to  be  balllstlcally  tolerant  to  a 
7.62  mm  projectile  at  2700  feat  par  second  (fps).  Because  of  the  high  cost  of 
shooting  the  seats  for  production  acceptance,  a  coupon  test  using  representative 
materials  was  developed .  The  test  demonstrated  that  the  coupon  failed  to  provide 
the  required  ballistic  tolerance. 


The  test  results  wera  contradictory  to  all  of  the  ballistic  experience  accu¬ 
mulated  up  to  that  time  and  an  investigation  into  the  reason  for  failure  was 
undertaken.  The  seat  and  coupons  consisted  of  a  boron-carbide  ceramic  plate 
bonded  to  a  Kevlar  frame.  The  first  step  was  to  determine  whether  the  Kevlar  or 
the  ceramic  was  at  fault.  Since  there  is  no  mechanical  property  or  combination 
of  properties  which  can  accurately  predict  ballistic  tolerance,  coupons  must  be 
subjected  to  actuel  weapons  fire  to  determine  the  ballistic  tolerance, 

II.  THE  Vsn  TEST. 

In  order  to  measure  the  ballistic  protection  afforded  by  an  armor  material, 
a  V50  test  is  performed.  Vjq  is  defined  as  that  projectile  velocity  which 
results  in  complete  penetration  SOX  of  the  time  and  partial  penetration  50%  of 
the  time.  The  definitions  of  complets  and  partial  penetration  depend  on  the 
ballistic  protection  criteria  being  used.  During  the  testing  the  Protection 
Ballistic  Limit  (PBL)  criterion  was  used.  Using  this  criterion,  a  complete 
penetration  occurs  whenever  a  fragment  or  fragments  are  ejected  from  the  back  of 
the  armor  with  sufficient  remaining  energy  to  pierce  a  "witness  plate".  The 
"witness  plate"  is  a  thin  sheet  of  aluminum  alloy  placed  6  inches  behind  and 
parallel  to  the  armor  plate.  If  light  can  be  seen  through  punctures  in  the 
witness  plate,  the  penetration  is  complete.  If  not,  the  penetration  1b  partial. 
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Thu  V5q  definition  results  from  the  fact  that  a  bullet  velocity  sufficent  to 
penetrate  one  coupon  may  result  in  only  a  partial  penetration  in  another  coupon. 
The  relationship  between  projectile  velocity  and  the  ballistic  tolerance 
satis fit's  the  mathematical  conditions  of  a  probability  distribution.  For  low 
projectile  velocities  ths  probability  of  a  complete  penetration  approaches  zero, 
for  high  velocities  the  probability  of  a  complata  penetration  approaches  one. 
Between  those  extremes  of  valocity  the  probability  increases  with  increasing 
velocity.  When  that  general  model  describes  ths  physical  events,  probability  of 
penetration  can  be  treated  as  s  probability  distribution  and  is  usually  described 
as  a  Gaussian  or  normal  distribution. 

The  procedure  to  experimentally  determine  V50  is  as  follows:  The  flrat  round 
shall  be  loaded  with  an  amount  of  propellant  calculated  to  give  the  projectile  e 
velocity  of  2750  (ips)  for  the  apecified  ballistic  limit  of  2700  (fpe)  •  Bach 
succeeding  round  shall  ba  loaded  with  an  amount  of  propellant  calculated  to 
produce  a  velocity  change  of  25  to  50  fps.  The  criteria  to  detarmlna  whether  an 
increase  or  decrease  of  velocity  is  required  is  as  follows: 

a.  If  the  preceeding  valocity  resulted  in  a  partial  penetration,  the 
charge  will  be  increased  to  produce  a  velocity  increase. 

b.  If  the  preoaedlng  valocity  resulted  in  a  complata  penetration,  the 
charge  will  be  decreased  to  produce  a  valocity  dacraaaa. 

c.  A  minimum  of  six  shota  will  ba  required  to  determine  each  V3Q.  The 
V50  is  equal  to  ths  avarsga  of  six  Impact  velocities  comprising  the  three  lowest 
velocities  resulting  in  complete  penetration  and  the  three  highest  velocities 
resulting  in  partial  penetration.  Additional  shots  srs  permitted  if'  after  six 
impacts  throe  complete  penetrations  and  three  partial  penetrations  have  not  been 
achieved. 

III.  PROBUM  WITH  THE  Vsn  TEST. 

Table  I  1? ununarizeu  the  evidence  which  suggests  that  the  assumption  of 
homogeneity  may  not  ba  correct. 


TABLE  I 


TEST  1 


2742 

Partial 

2776 

Partial 

2792 

Partial 

2849 

Complete 

2822 

Complete 

2804 

Complete 

TEST  2 


2740 

Partial 

2779 

Complata 

2755 

Complete 

2741 

Complata 

2702 

Complete 

2687 

Complete 
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ttoth  testa  were  drawn  from  a  single  lot  of  ceramic  and  Kevlar.  Thus  the  two 
tests  should  have  yielded  near  identical  V50  values.  However,  they  ahowud 
significantly  different  ballistic  tolerance  levels.  The  teBt  apparatus  was 
checked  and  reehecked  to  make  certain  that  the  fault  did  not  lie  with  the  test 
equipment.  No  fault  with  the  equipment  could  be  found. 

Throughout  the  testing  similar  incidents  occurred,  incidents  which  suggested 
that  there  may  be  some  defective  panels  either  in  a  single  lot  of  Kevlar  or  a 
single  lot  of  ceramic.  Put  another  way,  the  testing  suggested  thet  e  single  lot 
of  Kevlar  or  ceramic  was  not  homogenous.  There  could  be  panele  in  a  lot  which 
have  extremely  poor  ballistic  tolerance,  much  lower  than  had  been  evident 
previous  to  the  March  test. 

It  is  interesting  to  note,  that  the  nature  of  information  that  a  complete 
penetration  provides  is  not  tha  same  as  that  provided  by  a  strength  teat  where 
the  stress  at  rupture  ia  of  concern.  In  the  strength  test  a  defective  specimen 
ia  noticeable  by  its  unusually  low  rupture  strese.  The  streee  at  rupture  can 
be  plotted  and  those  apacimene  with  vary  low  rupture  atrangth  can  be  grouped 
and  investigated  further  or  eliminated  from  the  data  bate.  However,  a  velocity 
of  a  complete  penetration  does  not  provide  tha  aame  type  of  data.  A  complete 
penetration  at  a  specified  velocity  only  ehowa  that  tha  coupon  will  not  atop  a 
bullet  at  that  velocity.  It  does  not  provide  any  information  on  tha  bullet 
velocity  tha  coupon  will  stop.  Thua,  it  is  necessary  to  take  like  coupons  and 
continue  to  shoot  until  tha  bullet  la  stopped.  Lika  coupons  are  assumed  to  be 
coupons  made  from  a  single  lot  of  Kevlar  and  a  single  lot  of  ceramic. 

If  there  era  soma  defects  within  a  lot  of  Kavlar  or  ceramic  as  suggested 
by  tha  Investigation  than  the  question  Is,  "Is  the  V50  test  still  a  reasonable 
test  to  use  during  production  acceptance  testing?". 

1V •  THE  OPERATING  CHARACTERISTICS  (PC)  CURVE . 

It  became  obvious  during  the  course  of  the  investigation  thet  an  OC  Curve 
had  to  be  developed  for  this  armor  material.  But  again,  tha  fact  that  there 
may  b«  defects  in  a  lot  will  affect  tha  data  for  the  OC  Curve.  To  complicate 
matters  the  lots  of  each  material  are  of  limited  else.  A  lot  of  Kevlar  will 
make  only  12  teat  coupons  while  a  lot  of  Boron-Carbide  will  make  28.  How  then 
should  the  test  be  designed  to  determine  the  OC  Curve  that  will  screen  out  the 
effect  of  the  defectives? 

V.  CONCLUSION. 

This  author  poses  the  following  two  questions  to  the  27th  Conference  on  the 
Design  of  Experiments  in  Army  research! 

a.  Civan  that  a  lot  of  coupons  may  contain  soma  def actives  ia  a  Vjq 
teat  appropriate  for  production  acceptance? 

b.  How  can  a  tast  be  designed  to  determine  the  OC  Curve  that  minimises 
the  presence  of  defects? 
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ABSTRACT.  Oonsider  the  problem  of  evaluating  the  relative  merits  of 
several  drugs  that  hold  promise  for  treating  a  certain  disease.  (Of 
course ,  one  or  more  of  these  drugs  oould  be  placebos  or  currently  standard 
controls.)  Suppose  that  these  drugs  are  administered  to  a  set  of  patients 
similarly  affected  by  the  (disease  of  interest ,  and  that  the  ef feats  of  the 
drugs  are  evaluated  by  recording  repeated  observations  on  the  patients 

over  a  fixed  period  of  time.  Within  this  framework  there  are  many  questions 

\ 

of  interest,  For  example,  do  the  patient  groups  react  to  the  several 

I 

drugs  in  a  similar  fashion  over  timely  If  the  drugs'  effects  do  differ, 
which  of  the  drugs  aohieve  the  greatest  degree  of  control  over  the  disease? 
Which  of  the  drugs  reach  their  peak  effectiveness  most  rapidly?  The 
purpose  of  this  talk  is  not  to  disouss  the  relative  merits  of  how  to  rank 
for  suoh  two-way  layout  problems  (i.e.,  within-blocks  versus  between- 
blocks  rankings),  but  to  instead  consider  the  more  basio  question  of  What 
to  rank  to  best  answer  the  questions  of  interest.  Several  rank-like 
approaches  to  some  of  the  mentioned  problems  are  discussed. 
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1.  INTRODUCTION.  Since  the  pioneering  work  of  Friedman  (1937), 

Kendall  and  Babington-Smith  (1939),  and  Kruskal  and  Wallis  (1952)  there 
has  been  a  steady  flow  of  research  activity  in  the  area  of  nonparametric 
procedures  for  the  one-  and  two-way  layout  settings.  However,  there  has 
been  very  little  progress  in  the  development  of  satisfactory  nonparametric 
procedures  for  analyzing  data  involving  three  or  more  factors.  In  this 
paper  we  discuss  a  direction  for  future  research  that  could  help  alleviate 
at  least  one  of  the  impediments  to  suoh  progress.  Our  approach  hinges  on 
the  fact  that  the  intuitive  criteria  utilized  by  nonparametria  researchers 
differ  between  the  one-  and  two-way  layout  settings,  and  that  this  should 
suggest  an  even  different  set  of  criteria  is  necessary  for  three  or  more 
factor  experiments.  We  begin  by  considering  the  most  common  concerns 

put  forth  in  the  development  of  nonparamo  trio  procedures  for  the  one-  and 
two-way  layout  settings. 

2.  ONE-WAY  LAYOUT.  Let  X^  1*1,..,, n^  and  j*l, be  k 
independent  random  samples  from  continuous  distributions  with  distribution 
functions  fj(x)  =  F(x-tj),  j*l,,..,k,  Where  FCO  is  unspecified.  Most  of 
the  interest  for  this  setting  has  been  with  developing  effective  hypothesis 
tests  of  Hg :  [t^a.. where  the  word  effective  has  generally  been 
related  to  power  considerations  for  suoh  tests. 

What  has  been  the  primary  motivation  behind  the  different  approaches 
to  providing  "effective”  tests  for  these  one-way  layout  data?  It  has 
certainly  not  been  an  inclination  to  use  different  ways  to  "nonparameterize" 
the  information  in  the  k  samples.  I  think  that  it  is  safe  to  say  that 
virtually  every  nonparamatrio  one-way  layout  test  has  as  its  first  step 
the  replacement  of  the  original  observations  by  their  combined  camples 
ranks  from  least  to  greatest.  That  1b,  there  1b  no  distinction  between 
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competing  procedures  on  the  basis  of  what  ranking  scheme  is  used  in 
extracting  the  "nonpar ametric"  information  from  the  data — they  all  use 
combined  samples  ranks  of  the  original  observations. 

What  then  distinguishes  between  the  procedures?  Of  primary  importance 
has  been  the  desire  to  design  a  test  that  would  be  especially  powerful 
against  either  (i)  a  particular  parametric  model  or  (ii)  a  general  class 
of  alternative  hypotheses  of  interest.  Thus,  for  example,  the  parametric 
motivation  in  (i)  led  researchers  to  develop  such  criteria  as  looally 
most  powerful  or  asymptotically  most  powerful  rank  tests  as  disoussed, 
for  instance,  in  Randles  and  Wolfe  (1979).  Examples  of  such  tests  are 
the  one-way  layout  normal  scores  procedures. 

Interest  in  the  o'lass  of  alternatives  approach  in  (ii)  has  resulted 
in,  among  otherB,  the  test  procedures  proposed  by:  Kruskal  and  Wallis 
(1952)  for  general  alternatives  of  the  form  Rj, :  [t^t^  for  at  least  one 
i^j]i  Jonokheere  (1954)  and  Terpstra  (1952)  for  ordered  alternatives  of 
the  form  H, :  Ct^<.  .  .<rk,  with  at  least  one  Btriot  inequality] ;  and,  most 
reoently,  Hack  and  Wolfe  (1981)  for  umbrella  (quadratic)  alternatives 
of  the  form  H^:  [t-^.  .  .<1^  i  Tg,  1  Tx,+i  »  with  at  least  one  strict 

inequality],  where  fc,  the  peak  of  the  umbrella,  is  either  known  or 
unknown.  Other  applications  of  criterion  (ii)  have  been  carried  out  for 
the  k-sanple  slippage  problem  and  in  treatments  versus  control  settings. 
For  the  latter  model,  attention  has  also  been  given  in  Costello  and  Wolfe 
(1980)  to  collecting  the  treatments  observations  in  a  partially  sequential 
manner. 

3.  TWO-WAY  LAYOUT.  In  complicating  the  model  ty  adding  a  second 
factor  of  interest,  the  two-way  layout  also  introduces  another  area  of 
variability  in  the  proposed  approaches  to  related  problems.  Both  criteria 
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that  were  important  in  differentiating  between  one-way  layout  procedures 
remain  important  here  as  well.  Thus,  for  example,  we  have  the  general 
alternatives  procedure  due  to  Friedman  (1937)  and  Kendall  and  Babington-Smith 
(1939),  as  well  as  the  ordered  alternatives  approach  suggested  by  Page 
(1963),  all  of  these  dealing  with  the  situation  where  there  is  one 
(no  missing  data)  and  only  one  (no  replications)  observation  collected 
for  each  combination  of  the  two  factors.  General  alternatives  procedures 
for  the  case  of  zero  or  one  observation  for  each  factor  combination  have 
been  proposed  by  Durbin  (1951)  when  we  have  a  balanced  imcomplete  block 
design  and  by  Skillings  and  Mack  (1981)  for  a  more  general  case  of 
arbitrarily  missing  data.  For  replications  within  the  factor  combinations, 
general  alternatives  proaedureB  have  been  considered  by  Mack  and  SkillingB 
(1980)  and  Mack  (1981),  while  a  similar  procedure  for  ordered  alternatives 
was  studied  by  Skillings  and  Wblfe  (1977,  1978)  and  Skillings  (1980). 

However,  discussion  over  the  appropriate  procedure  to  use  in  a 
two-way  layout  setting  has  not  been  limited  to  the  factors  of  (i)  parametric 
model  or  (ii)  alternatives  of  interest,  as  has  been  the  case  in  the  one-way 
layout.  For  two-way  layout  data,  we  also  see  considerable  discussion  on 
a  very  basic  third  factor,  namely,  (iii)  how  to  rank  the  collected 
observations.  To  briefly  describe  this  discussion,  let  X^j ,  i=l,...,n 
and  j=l,.,.,k,  be  mutually  independent,  continuous  random  variables  with 
having  distribution  function  F^ (x)  =  F(x-tj-S^) ,  where  F( • )  is 
unspecified.  Thus  the  t's  represent  the  effects  of  the  various  levels 
of  one  of  the  factors  and  the  8's  represent  the  effects  of  the  various 
levels  of  the  second  factor.  (Note  that  an  additive  model  is  usually 
assumed.  The  problem  of  interaction  has  been  particularly  thorny  in 
nonparametric  statistics.)  In  discussing  tests  of  Hg!  . .=t^]  against 


a  particular  alternative  of  interest  in  this  two-way  layout  setting  we 
are  also  fared  with  the  problem  of  how  to  rank  the  data:  Do  we  rank 
observations  only  within  levels  of  the  second  factor  (i.e. ,  rank 

separately  for  each  i=l,...,n)  or  is  there  some  (appropriate) 
way  that  we  can  effectively  rank  all  kn  of  the  values  together,  as 
is  done  in  the  one-way  layout  setting?  If  this  joint  ranking  can  be 
legitimately  (without  comparing  apples  and  oranges)  accomplished  without 
undue  complication,  it  should  produce  reasonable  competitors  to  those 
based  on  ranking  only  within  the  levels  of  the  second  factor. 

All  of  the  two-way  layout  procedures  previously  mentioned  in  this 
paper  utilize  the  within- levels  ranking  scheme.  Hodges  and  Lehmann  (1962) 
were  pioneers  in  the  area  of  jointly  ranking  all  of  the  observations  when 
they  suggested  using  aligned  ranks  in  constructing  appropriate  conditional 
test  procedures.  Doksum  (1967)  and  Hollander  (1967)  considered  other  ways 
to  use  between  block  information  and  still  obtain  at  least  asymptotically 
distribution-free  tests.  Mehra  and  Sarangi  (1967)  studied  the  power 
properties  of  some  of  the  wi+hin-levels  ranking  procedures  relative  to 
those  based  on  joint  ranking  schemas.  The  verdict  on  how  in  rank  is  not 
unanimous. 

4.  SEVERAL  TWO-WAY  LAYOUT  GROUPS.  This  brings  us  to  the  actual 

title  of  the  talk,  namely,  comparing  several  groups  ii  a  two-way  layout 

setting.  For  j=l,...,k  and  i=l,...,nj,  let  be  mutually 

independent,  continuous  random  vectors  such  that,  for  each  fixed 

je{l,...,k},  the  vectors  (X^,. •  •  •• »^n  are 

identically  distributed  with  joint  distribution  function  Fj  (x-^ , . . .  jX^) 

and  median  vector  (t., . t.  ). 

jl  jm 

In  this  section  we  consider  general  distribution-free  approaches 
to  constructing  hypothesis  tests  about  the  t  vectors.  First,  however,  it 
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might  be  helpful  to  discuss  the  applied  setting  that  led  to  our  interest 
in  such  problems.  Consider  k  different  drugs  that  are  potentially 
useful  for  treating  a  certain  illness.  (One  or  more  of  the  drugs  could 
certainly  be  control-standards  or  control-placebos . )  These  drugs  are 
administered  to  patientB  with  the  prescribed  illness  and  the  effects  of 
the  drugs  are  recorded  over  a  specified  period  of  time.  That  is,  each 
administration  of  one  of  the  drugs  to  a  patient  results  in  repeated 
(dependent)  measurements  on  the  same  subject  over  time.  We  are,  of  course, 
interested  in  potential  treatment  effect  differences  among  the  k  drugs 
ovw  the  involved  time  period. 

To  set  this  problem  in  our  stated  model,  we  take  X^s  to  be  the 
measurement  at  the  sth  time  point  for  the  ith  subject  being  treated  with 
the  jth  drug.  We  thus  have  n^  patients  taking  the  jth  drug,  j -1 , . . . ,k, 
and  being  evaluated  at  m  distinat  time  points,  and  our  interest  is  in 
making  inferences  about  the  relative  treatment-time  effects  of  the  k  drugs. 

Similar  problems  in  the  context  of  testing  for  agreement  between 
two  groups  of  judges  have  been  considered  by  Schucany  and  Frawley  (1973), 

Li  and  Schucany  (1975),  Schucany  and  Beckett  (1976),  and  Hollander  and 
Sethuraman  (1978).  The  Schucany-Frawley-Li  test  is  based  on  the  average 
value  of  an  appropriate  series  of  Spearman  correlations'  between  rankings 
from  one  group  of  judges  and  rankings  from  the  other  group  of  judges. 
However,  Hollander  and  Sethuraman  suggested  some  possible  problems  in 
the  consistency  class  and  designated  null  hypothesis  for  the  Schucany- 
Frawley-Li  procedure,  and  they  proposed  a  solution  based  on  a  conditionally 
distribution-free  permutation  test  utilizing  the  Mahalanobis  D  statistic. 

Regardless  of  the  relative  merits  of  these  competing  procedures 
for  the  problem  of  two  groups  of  judges,  it  is  not  obvious  how  either  of 
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them  would  be  naturally  extended  to  either  the  more  general  problems 
posed  by  competing  drug  studies  or,  even  in  the  context  of  their  problem, 
to  more  than  two  groups  of  judges.  For  these  reasons  we  approach  the 
drug  evaluation  problems  from  a  different  viewpoint  in  this  paper. 

We  have  previously  noted  that  power  considerations  against  certain 
classes  of  alternatives  have  been  the  primary  motivations  behind  the 
development  of  most  distribution- free  one-way  layout  test  procedures j 
that  is,  how  to  extract  the  important  information  from  the  agreed-upon 
ranking  method  has  been  paramount.  To  this  criterion  is  added  the  problem 
of  how  to  rank  the  collected  data  in  a  more  complicated  two-way  layout 
setting.  When  we  extend  this  one  step  more  to  the  consideration  of  several 
groups  in  a  two-way  layout  setting,  we  suggest  that  a  third  criterion, 
namely  that  of  what  to  rank,  should  be  given  at  least  as  much  (and 

s 1 

probably  most)  attention  in  developing  appropriate  teBt  procedures. 

Thus  instead  of  automatically  presuming  that  our  tests  should  be  based 
on  some  function  of  some  method  for  ranking  the  sample  observations 
themselves,  perhaps  it  would  be  beneficial  to  at  least  consider  if  there 
are  other  quantities  that  could  be  effectively  ranked  to  address  our  questions. 
Such  rank-like  (i.e.,  ranking  of  quantities  other  than  the  collected  data 
values)  techniques  have  been  proposed  by  Fligner  and  Killeen  (1976) , 

Fligner,  Hogg  and  Killeen  (1976),  Brofitt,  Randles  and  Hogg  (1976),  and 
Smith  and  Wolfe  (1977),  and  are  discussed  in  Randles  and  Wolfe  (1979). 

Similar  ideas  have  also  been  utilized  by  Kboh  (1972)  in  dealing  with  the 
use  of  nonparametric  methods  in  a  two-period  change-over  design. 

Now,  returning  to  our  problem  of  evaluating  several  drug  treatment 
groups  in  a  two-way  layout  setting,  we  demonstrate  this  idea  of  the 
importance  of  what  to  rank  through  a  series  of  examples  dealing  with 
different  alternatives  of  interest. 

339 

. . 


Example  4.1.  Suppose  we  wish  to  know  which  of  the  drugs  under 
consideration  achieves  the  greatest  peak  effectiveness.  This  could 
correspond  to  either  highest  or  lowest  measurement  values,  depending  on 
the  nature  of  the  data  being  collected.  Fbr  purposes  of  this  paper,  we 
will  take  large  values  to  mean  good  effectiveness  of  a  drug.  Letting 
XJj  =  maximum  (X^),  for  1*1,,,., nj  and  j=l,... ,k,  represent  the  maximum 
measurement  value  achieved  by  the  ith. subject  on  the  jth  treatment  drug, 
we  see  that  the  null  hypothesis  of  interest  here  could  be  taken  to  be 
Hgi  [Oj*. . .=6£],  where  9*  represents  the  median  of  the  distribution 

of  X?j ,  for  1*1,..., m.  [Note  that  6?  is  analogous  to  max  t,  .  ] 

3  3  *  a  a  1<s<m  3 

Appropriate  procedures  for  testing  Hq!  [9^..  ,*e£]  oan  be  Eased  on  the 


k  , 


joint  rankings  of  the  E  n4  Xjj  values.  What  particular  method  of 

j!l  ^ 

evaluating  this  ranking  information  should  be  used  will  still  depend 


on  the  alternative  to  Hq  that  is  of  interest.  For  example,  if  the 

treatment  drugs  are  suoh  that  H^s  [©*<_. .  .<9*,  with  at  least  one  striot 

inequality]  is  appropriate,  then  we  could  apply  the  Jonokheere  (1954) 

A 

procedure  to  the  X^'s,  While  for  general  alternatives  the  Kruskal -Wallis 

(1952)  would  be  preferred.  The  main  point  is  that  such  procedures  would 

be  applied  to  the  X^ 's,  not  the  original  data. 

Example  4.2.  In  this  example  we  would  like  to  evaluate  which  of 

the  drugs  is  quickest  to  achieve  its  peak  effectiveness.  Thus,  letting 

"  maximum  t.  and  taking  t.  to  be  the  time  point  for  which 
3  l<8<m  ^ B  J 

a"”£j,  we  are  here  interested  in  testing  HQ!  [^=...=^3  against 
general  or  ordered  alternatives,  for  example.  To  construct  a  distribution- 
free  test  for  this  setting,  we  can  again  use  the  rank-like  idea.  Let 
X^j  be  as  defined  in  Example  4.1  and  oonsider  the  sample  time  points 
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where  the  various  drugs  reach  their  greatest  effects  (as  measured  by 

the  xV,.  values)  on  the  patients  to  which  they  were  given.  Setting 

Njs  =  [number  of  patients  given  the  jth  drug  that  achieved  their 

maximum  measurement  value  at  the  sth  time  point], 

for  j«l,...,k  and  a*l,...,m,  we  could  than  test  Hqj  [tj*. . against 

either  general  or  ordered  alternatives  by  applying  an  appropriate  procedure 

for  testing  equality  of  multinomials  to  the  k  sets  of  counts  -V  * 

for  jsl,...,k.  (Mote  that  statistics  other  than  the  maximum  (Xj.J’s 

l<s<m  138 

could  be  used  to  indicate  when  the  peak  effectiveness  of~&  drug  is 
reached.  For  example,  the  peak-pioker  employed  by  Mack  and  Wolfe  (1981) 
could  be  used  effectively  here  as  well.) 

Example  4.3.  Consider  the  problem  discussed  in  Exanple  4.2  but 
with  the  further  assumption  that  we  know  the  magnitude  of  the  peak 
effectiveness  is  the  same  for  the  k  drugs.  Thus,  in  this  example  we 
want  to  test  HQ i  [t^« , . . =tu]  under  the  additional  information  that 
^1*. * ,s^k*  ^  8U°h  a  situation  we  are  able  to  describe  an  exact  distribution* 

free  rank-like  test  that  is  a  competitor  to  the  approximate  multinomial 
test  discussed  in  Example  4.2.  Let  represent  the  slope  of  the  line 
connecting  X^  and  X^  *  naximum  X^s,  for  i»l,. . . ,nj  and  j*l,...,k. 


X?,-X, 


(That  is,  «*  T^j-T  ' "»  wher*  Tij  the  time  point  corresponding  to 

X*^  and  Tq  is  the  time  point  for  the  initial  measurement.)  An  appropriate 
(depending  on  alternatives  of  interest)  nonparamatrlc  distribution-free 
one-way  analysis  of  varianoe  procedure  oould  then  be  applied  to  the  k 
sets  of  estimated  slopes.  Such  a  test  would  be  exactly  distribution-free 
under  the  hypothesis  of  no  differenoe  in  time  effectiveness  for  the 
k  drugs  and  would  be  especially  powerful  at  detecting  differences  in  the 
peak  time  points  . ,tk> 
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Actually  the  rank-like  technique  discussed  in  Exairple  4.3  could  be 
useful  even  if  we  do  not  know  that  all  the  drugs  have  a  common  peak 
effectiveness.  However,  we  would  then  have  to  be  willing  to  aooept  both 
early  peaking  and  larger,  but  slower-achieved  peaks,  as  indicative  of  an 
effective  drug,  since  either  of  these  occurrences  in  the  data  would  lead 
to  large  estimated  t4,i  slope  values. 

Example  4.4.  As  a  final  example  consider  the  problem  of  evaluating 
whether  the  overall  time-collected  reactions  of  patients  are  similar  for 
the  k  drugs  in  the  study.  (The  settings  discussed  in  Examples  4.1,  4.2, 
and  4.3  address  particular  aspects  of  this  problem.)  To  use  a  rank-like 
procedure  for  this  general  question,  we  must  first  settle  on  a  within- 
subjects  statistic  that  is  representative  of  our  interest  in  the  time-collected 
data.  Bor  example,  we  might  wish  to  assume  a  straight  line  regression 
relationship  between  the  measurements  being  oolleoted  and  the  times  at 
vhich  the  data  are  obtained.  If  so,  then  a  statistic  such  as  an  estimator 
for  the  elope  of  the  regression  line  would  be  a  logical  oandidato  for 
comparisons  between  the  k  drug  groups.  That  is,  we  would  obtain  estimates, 

ajl  k 

py  j ,  of  the  slopes  associated  wish  each  of  the  I  na  individuals  in  the 
13  j-1  3 

study.  (What  method  of  estimation  (e.g. ,  least  squares,  median  of  all 

sample  slopes,  as  disousssd  in  Section  9.3  of  Hollander  and  Wolfe  (1973), 

A  A 

etc.)  is  used  to  obtain  the  values  is  not  important  for  maintaining 
the  distribution- free  property  of  the  process.  It  is  only  necessary  that 
the  same  method  be  used  for  all  the  individuals.  Of  course,  the  choice  of 
estimation  criterion  could  indeed  have  an  effeot  on  the  power  properties 
of  the  resulting  distribution- free  test,)  After  obtaining  these  individual 
slope  estimators,  we  would  then  proceed  as  in  Example  4,3  by  applying 
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an  appropriate  (depending  on  alternatives  of  interest)  nonparametric 
distribution-free  one-way  analysis  of  variance  procedure  to  the  k  groups 
of  condensed  data. 

If  a  regression  model  more  complex  than  a  straight  line  is  necessary 
to  relate  the  sample  observations  and  the  time  pointB  at  which  the  data 
are  collected ,  a  similar  approach  can  be  used  to  develop  an  appropriate 
distribution-free  test.  All  that  is  required  is  some  aunrnary  measure  to 
represent  this  regression  model  for  each  of  the  individual  subjects. 

So  long  as  the  same  statistic  is  computed  for  eaoh  of  the  individuals , 
the  resulting  analysis  of  variance  test  will  be  nonparametric  distribution- 
free  for  the  null  hypothesis  of  no  difference  in  regression  time  effects 
for  the  k  drugs.  This  will  remain  the  case  no  matter  how  complex  we 
make  either  the  regression  model  or  the  swroaxy  statistio.  (Similar 
approaches  could  also  be  taken  even  if  there  were  multiple  or  missing 
observations  for  eaoh  individual  at  soma  time  points.) 

5 .  DISCUSSION.  The  primary  intent  of  this  paper  has  not  been  to 
propose  and  study  a  new  test  for  a  given  problem,  but  rather  to  re-emphasize 
the  flexibility  that  is  available  in  constructing  nonparametric  distribution- 
free  tests  of  hypotheses.  Keeping  in  mind  the  well-established  advantages 
of  tailoring  tests  to  alternatives  of  interests  and  the  potential  gains 
from  consideration  of  different  methods  of  ranking  the  actual  collected  data, 
we  have  suggested  that  an  even  more  basic  question  of  what  to  rank  can 
play  an  important  role  in  more  complicated  problems  such  as  comparing 
several  groups  in  a  two-way  layout  setting.  The  use  of  such  rank-like 
(i.e,,  ranking  something  other  than  the  original  data  points)  would  also 
seem  to  have  both  appeal  and  merit  for  other  problems,  such  as  with 
multivariate  data,  Where  the  usual  nonparametric  approaches  have  proven  to 
be  less  than  totally  effective. 
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A  MATHEMATICAL  BASIS  FOR  TRACKING 
MANEUVERING  AIRCRAFT  WITH  DOPPLER  RADAR 


Donald  W.  Rankin 
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ABSTRACT.  Continuous  Wave  Doppler  Acquisition  Radars  have  several  distinct 
TOTtfltlJes  when  employed  against  aircraft.  Among  them  are  (1)  Illumination  of 
the  target  with  great  power  and  (2)  elimination  of  unwanted  returns  from  sta¬ 
tionary  objects,  achieved  by  means  of  a  suitable  band-stop  filter. 

However,  Information  arrives  as  azimuth  and  radial  velocity,  a  form 
unsuited  to  coordinate  transformation.  First,  range  must  be  computed.  Essen¬ 
tially,  this  Is  a  problem  In  numerical  Integration. 

Traditionally,  tracking  algorithms  simplify  this  Integration  problem  b,y 
assuming  that  the  acceleration  and  velocity  vectors  are  coll  Inear.  The 
assumption  holds  quite  well  In  the  case  of  ballistic  missiles  (computing  In  an 
inertial  coordinate  system),  but  for  maneuvering  aircraft  Is  manifestly 
erroneous, 

This  paper  reexamines  the  equations  of  motion  of  maneuvering  aircraft  with 
a  view  toward  finding  assumptions  upon  which  to  base  a  mathematical  model  for 
an  efficient  tracking  algorithm, 

An  algorithm  Is  developed,  based  upon  the  assumption  of  constant  aircraft 
turn  rate.  Preliminary  results  against  synthesized  targets  are  most  satis¬ 
factory. 

I.  INTRODUCTION 


During  a  recent  test,  the  author  was  assigned  to  evaluate  the  software 
associated  with  the  range  tracking  algorithm  of  a  particular  continuous  wave 
(Doppler)  acquisition  radar,  which  was  exhibiting  unexplained  anomalies.  In 
the  course  of  the  Investigation,  It  became  Increasingly  apparent  that  the 
subject  of  tracking  algorithms  Is  not  well-documented  and  that  sound,  basic 
reference  materiel  Is  not  always  readily  available  to  the  composer. 

The  evaluation  led  to  the  Identification  of  a  single  critical  target  per¬ 
formance  parameter,  hitherto  apparently  unnoticed,  upon  which  a  mathematical 
development  properly  could  be  based.  When  this  critical  parameter  Is  assumed 
to  be  constant,  the  ensuing  development  provides  a  rational  basis  for  select¬ 
ing  an  efficient  mathematical  model  which  has  demonstrated  excellent  tracklnq 
performance  during  computer  simulations. 
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In  view  of  all  this,  It  was  decldod  to  Include  a  discussion  of  certain 
fundamental  principles.  It  Is  hoped  that  some  use  may  be  found  by  others  when 
composing  or  evaluating  tracking  algorithms. 

a.  REMARKS  CONCERNING  OATA  HANDLING 

Let  us  define  two  terms  for  which  we  shall  find  frequent  use: 

(1)  Track  file  data  —  left-over  data,  processed  or  not,  still  available 
In  the  computer 

(2)  Raw  data  —  Incoming  data,  not  yet  processed 

Now,  If  we  place  great  confidence  In  the  accuracy  of  the  raw  data,  we 
simply  use  It  for  whatever  purpose  we  wish  and  pay  little  attention  to  the 

track  file  data.  Unfortunately,  this  Is  rarely  the  case.  Usually,  the  raw 

data  Is  contaminated  by  noise  and  possibly  other  errors. 

In  the  latter  event,  the  only  criterion  for  evaluating  the  raw  data  lies 
within  the  track  file  data.  Two  methods  commonly  are  use  —  often  combined: 

(1)  The  "window.11  From  considering  performance  characteristics  of  the 
target  being  tracked,  It  may  be  possible  to  say  that  the  raw  data  cannot  vary 
from  the  track  file  data  by  more  than  a  specified  amount,  say  w.  If  It  does, 
It  Is  presumed  to  come  from  a  different  source. 

(2)  The  "filter.11  It  Is  assumed  that  some  value  which  lies  between  the 

raw  data  and  the  track  file  data  Is  a  better  estimate  than  either  of  them 
alone.  As  an  example,  let  us  supoose  the  track  file  contains  computed  radial 
velocity  and  acceleration,  ft  and*.  Some  At  seconds  later,  raw  radial  veloc¬ 
ity,  ftm,  arrives.  On  the  basis  of  track  file  Information  alone,  we  could  have 

predicted  a  value  for  radial  velocity  at  this  time: 

ftp  ■  ft  +  At  "ft 


Now,  If 


I  Am  ■  Apt  >  w 

It  Is  said  that  ftffl  falls  outside  the  window  and  therefore  Is  not  associated 
with  this  track.  If,  however,  falls  within  the  window,  a  filtered  (or 

smoothed  or  updated)  value  of  the  radial  velocity  Is  computed  by 

ft  ■  kftm  +  (1  -  k)  ftp 
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(1) 


An  exactly  equivalent  expression  Is 

ft  -  ftp  +  k(ftm  -  ftp) 


Note  that  In  order  to  Insure  that  the  new  value  of  ft  lies  between  ftm  and  ftp, 

It  Is  necessary  that  0  <  k  <  1,  (It  should  be  noted  In  passing  that  some 
command  and  control  systems  may  employ  a  value  of  k  >  1,  But  In  every 
Instance  of  this,  the  filter  Is  being  used  for  some  purpose  other  than  smooth¬ 
ing.) 

b,  MISSILE  OR  AIRCRAFT?  THE  PROPER  USE  OF  ACCELERATION 

Probably  there  exists  no  such  thing  as  a  satisfactory  universal  tracking 
algorithm.  It  seems  self-evident  that  a  procedure  which  accurately  tracks 
ballistic  missiles  might  be  worthless  against  helicopters.  Each  algorithm 
must  be  tailored  to  a  specific  purpose,  Ignoring  those  problem  areas  whose 
probability  of  occurrence  must  be  quite  small. 

One  Indispensable  step  Is  to  define  areas  and  to  simplify  procedures  by 
making  certain  arbitrary  a  priori  assumptions,  based  upon  knowledge  of  per¬ 
formance.  Thus,  an  algorithm  to  track  helicopters  might  well  assume  that  no 
velocities  will  be  encountered  In  excess  of  100  m/sec.  Such  an  assumption  In 
a  missile  tracker,  however,  would  be  a  serious  error. 

Suppose  we  wish  to  compose  an  algorithm  for  tracking  ballistic  missiles. 
Assuming  acquisition  outside  the  atmosphere  after  thrust  Is  spent,  accelera¬ 
tion  will  be  zero  (except  for  gravity,  the  effect  of  which  can  be  removed  by 
computing  In  a  suitable  moving  coordinate  system).  That  Is,  velocity  will  be 
virtually  constant  and  the  acquisition  arithmetic  becomes  quite  easy.  At 
atmospheric  re-entry,  we  assume  that  drag  produces  a  negative  acceleration 
which  changes  very  slowly  and  Is  directed  along  the  longitudinal  axis  of  the 
missile  (l.e,,  parallel  to  the  velocity  vector).  Acceleration  Is  obtained  by 
differencing;  hence,  the  value  Is  "old."  but  easily  can  be  updated.  This 
updated  value  Is  used  to  predict  velocity  (and,  In  turn,  position).  This 
results  In  a  vary  simple  and  satisfactory  algorithm  which  should  produce 
accurate  results. 

Passing  to  the  problem  of  tracking  aircraft,  It  can  be  shown  that  the 
foregoing  missile-tracking  algorithm  Is  unsuitable.  Let  us  examine  the 
assumptions.  First,  the  assumption  of  zero  acceleration  limits  acquisition  to 
aircraft  In  virtually  straight,  unaccelerated  flight.  For  acquisition,  the 
assumption  of  constant  acceleration  Is  far  less  restrictive  and  therefore 
preferable. 

Second,  to  assume  that  the  acceleration  and  velocity  vectors  are  parallel 
Is  completely  untenable,  The  aircraft  maneuver  most  likely  to  occur,  which 
produces  measurable  acceleration,  Is  the  simple  turn.  In  a  turn  of  constant 
angular  rate  (which  all  pilots  try  hard  to  achieve),  the  aircraft  flies  In  a 
circle  with  the  acceleration  vector  directed  towards  Its  center.  Thus,  the 
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acceleration  and  velocity  vectors  are  directed  90  degrees  apart  and  are  said 
to  be  out  of  phase.  A  knowledge  of  one  Is  of  little  use  In  predicting  the 
other.  We  are  not  left  helpless  In  this  situation,  however. 

Before  proceeding  further,  let  us  pause  to  coin  a  new  word  and  to  define 
It.  Let  the  time  derivative  of  the  acceleration  be  called  the  ACC'RATE.  It 
will  have  the  dimension  m.sec’3  or  some  equivalent.  Suppose  we  Imagine  that 
an  aircraft  Is  flying  In  a  circle  at  a  constant  speed  and  that  we  are  observ¬ 
ing  It  from  a  point  sufficiently  far  away  that  azimuth  can  be  Ignored.  Obvi¬ 
ously,  the  relative  velocity  varies  as  cos  e,  with  e  being  the  central  angle. 
Now  the  derivative  of  the  cosine  Is  just  the  negative  of  the  sine,  so  that  the 
relative  acceleration  varies  as  -sin  e.  Extending  the  process  one  more  step, 
It  Is  seen  that  the  relative  acc'rate  varies  as  -cos  e,  What  this  means  Is 
that  the  ratio 

afic.'r&Lfl 

velocity 

Is  constant  and  forever  negative  (or  zero)! 

In  the  computer,  derivatives  are  estimated  by  taking  differences.  Letting 
triple  dots  Indicate  ace'rate,  we  find  that,  with  suitable  scaling, 

-  ftn  "  2^n-l  +  ^n-2  W 

Letting  q  denote  the  estimate  of  the  acc'rate  ratio,. 


Sid  .  V'.tf.n-J-lk-.S  . 


Now  Is  constant  and  therefore  q  nearly  so. 
ftn-l 

Treating  q  as  a  constant  enables  us  to  drop  the  subscript  and  write 

ftn  *  ^n-2  "  +  q)  (3) 


Equation  (3)  can  be  used  to  compute  q.  Or,  If  q  Is  known,  It  becomes  a  three- 
term  recurrence  relation  for  the  successive  values  of  ft.  It  should  be 
observed  that  equation  (3)  Is  Independent  of  the  units  In  which  at,  R,  and  ft 
are  expressed. 
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c.  COORDINATE  SYSTEMS 


It  seems  desirable  to  write  math  models  and  algorithms  which  will  operate 
In  two-  or  three-dimensional  cartesian  or  Inertial  systems  whenever  possible. 
The  data,  of  course,  must  be  amenable  to  transformation.  In  the  case  of 
Doppler  radar,  It  Is  not.  The  data  are  available  as  radial  velocity  and 
azimuth,  essentially.  Before  a  transformation  can  be  effected,  range  must  be 
determined  --  and  this  very  range  reduction  requires  that  a  track  file  be 
established  In  that  coordinate  system  In  which  we  find  ourselves. 

The  fact  that  we  necessarily  are  operating  In  polar  coordinates  produces 
an  apparent  outward  acceleration  which  depends  upon  both  the  range  of  the 
target  and  Its  relative  angle  of  approach.  The  effect  on  a  crossing  track  at 
short  range  can  be  appreciable.  For  example,  an  aircraft  at  a  range  of  6  km 
flying  a  straight-line  path  with  a  constant  speed  of  280  m.sec"1  at  a  relative 
radial  angle  of  60  degrees  appears  to  have  an  outward  acceleration  of  9.8 
m.sec"2  when  In  fact  there  Is  none.  This  phenomenon  Is  discussed  further  In 
section  V, 

d.  SOME  PRINCIPLES  OF  RADAR 

By  far  the  most  common  radar,  and  the  one  with  which  many  readers  will  be 
familiar.  Is  the  pulsed  radar.  A  short  burst  of  energy  Is  transmitted,  fol¬ 
lowed  by  a  long  period  of  silence.  If  during  this  listening  period  the 
receiver  detects  a  reflected  signal,  Its  time  delay  Is  measured  from  which 
range  can  be  computed. 

The  data  Is  reported  as  range  and  azimuth,  which  Is  already  a  perfectly 
good  two-dimensional  polar  coordinate  system.  Transforming  the  data  Into  any 
other  desired  coordinate  system  Is  very  easy. 

However,  when  the  radar  beam  Is  directed  at  a  very  low  angle,  many  reflec¬ 
tions  are  received  from  stationary,  ground-based  objects.  Returns  from  low- 
flying  aircraft  are  lost  In  a  welter  of  unwanted  targets.  In  fact,  a  pilot's 
standard  radar  avoidance  technique  has  always  been  to  fly  at  tree-top  level. 

In  an  effort  to  detect  these  low-flying  aircraft,  the  continuous  wave  (CW) 
radar  was  developed.  The  CW  radar  operates  on  the  following  principle.  There 
Is  neither  pulse  nor  listening  period.  Instead,  a  continuous  unmodulated 
carrier  wave  Is  radiated.  If  any  reflected  signals  are  detected,  they  are 
"beat"  against  the  transmitted  wave.  Reflection  from  a  moving  aircraft  will 
alter  the  frequency,  producing  the  well-known  Doppler  effect,  and  yielding  a 
measurable  beat-frequency.  A  band-stop  filter  can  be  applied  near  beat- 
frequency  zero,  eliminating  unwanted  reflections  from  stationary  targets. 

(Also  eliminated  are  returns  from  laterally-flying  aircraft,  since  their 
radial  velocity  will  be  near  zero,  but  these  are  of  less  Interest  than  those 
which  are  approaching!) 


Data  arrives  in  natural  units  of  velocity  and  azimuth  --  not  suitable  for 
transformation.  Range  first  must  be  computed.  Early  CW  radars  attempted  to 
recover  the  range  by  integration.  But  from  the  very  beginning,  integral  cal¬ 
culus  teaches  us  that  an  arbitrary  constant  may  be  added  to  any  Integral  with¬ 
out  affecting  the  validity  of  the  solution.  Evaluation  of  this  constant  of 
integration  Is  the  whole  burden  of  the  subject  of  definite  Integrals  and  often 
proves  to  be  difficult,  or  even  impossible  for  mid-flight  acquisition,  the 
problem  we  are  presently  faced  with. 

Clearly  we  must  get  some  bit  of  range  Information  from  somewhere.  Yet  the 
Doppler  radar's  sole  raison  d'etre  Is  the  Inability  of  other  radars  to  detect 
under  the  stated  conditions.  Some  device  Is  needed,  which  Is  Internal  to  the 
radar  Itself,  that  allows  range  to  be  determined. 

e.  RANGE  DETERMINATION  WITH  THE  DOPPLER  RADAR 

We  define  a  scan  as  that  period  during  which  the  radar  antenna  rotates 
through  360  degrees,  beginning  and  ending  at  some  known  reference  point. 

During  alternate  scans  (odd-numbered  ones,  let  us  say),  an  unmodulated 
carrier  wave  of  constant  frequency  Is  continuously  radiated.  The  Doppler 
velocity  Is  measured  In  the  standard  way.  We  may  call  these  scans  CW-scans. 

During  the  remaining  scans  (even-numbered,  of  course),  the  carrier  wave  Is 
modulated  by  Increasing  the  frequency  at  a  known  linear  rate.  Thus  the  fre¬ 
quency  can  be  said  to  have  a  constant  ramp.  Note  that  the  reflected  signal 
will  be  beat  against  a  frequency  farther  advanced  along  the  ramp,  causlnq  an 
apparent  reduction  in  Doppler  velocity  (for  approaching  targets).  How  much 
farther  along  the  ramp  Is  a  function  of  elapsed  time,  and  hence  of  i  ahge. 
Stationary  targets  will  appear  to  recede,  and  hence  can  be  filtered  out.  We 
may  call  these  scans  FM-scans. 

If  It  can  be  assumed  that  the  radial  velocity  Is  constant,  then  any  two 
consecutive  scans  will  produce  sufficient  data  that  range  (at  the  time  of  the 
FM  scan)  can  be  determined. 

However,  the  ability  of  an  aircraft  to  execute  simple  turns  enables  It  to 
produce  large  accelerations,  the  direction  of  which  with  respect  to  the  radius 
vector  (from  radar  to  aircraft)  can  be  quite  random.  Therefore,  any  assump¬ 
tion  of  constant  radial  velocity  Is  completely  untenable. 

Thus  Is  strongly  highlighted  the  basic  problem  which  any  algorithm  must 
solve.  It  Is  the  computation  or  prediction  of  radial  velocity  for  the  time  of 
an  FM  scan.  Only  when  a  satisfactory  solution  has  been  found  can  range  be 
computed . 
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Equation  (3)  Is  well-suited  to  this  purpose. 

The  concept  of  frequency  modulation,  however,  Introduces  a  new  problem 
which,  for  want  of  a  better  name,  we  shall  call  "FM  drop-out."  If  the 
returned  signal  Is  designated  alternately  as  CW  and  FM,  and  If  R  and  ft  Indi¬ 
cate  the  actual  range  and  radial  velocity,  then,  with  suitable  scaling, 

CW  ■  ft  and  FM  »  ft  -  hR  (4) 

The  constant  h  depends  upon  the  frequency  ramp  and  has  the  dimension  sec"1.  A 
representative  value  Is  h  ■  0.002  sec"1. 

When  R  Is  too  great,  or  ft  too  small,  the  resulting  FM  signal  will  not  pass 
the  band-stop  filter,  and  FM  drop-out  occurs.  The  situation  arises  at  very 
great  ranges,  with  wide  crossing  angles,  and  for  slow-moving  aircraft.  All 
these  cases  tend  to  fall  In  an  area  of  lesser  Interest. 

Too  great  a  value  of  h  also  aggravates  FM  drop-out  but  that,  of  course,  Is 
a  design  problem. 

II.  MATHEMATICAL  PRINCIPLES  AND  DEVELOPMENTS 
a.  THE  DEVELOPMENT  OF  q 

The  following  assumptions  are  made: 

An  aircraft  Is  In  a  turn  of  constant  rate,  d9/dt.  The  value  Is  taken  as 
positive  when  the  aircraft  Is  turning  to  Its  right. 

Range  Is  sufficiently  great  that  small  changes  In  azimuth  or  altitude  can 
be  Ignored  without  serious  error. 

Central  angle  e  -  0  when  aircraft  Is  approaching  radar  head-on. 

Aircraft  Is  moving  at  a  constant  speed  v. 

Under  these  conditions,  If  R  denotes  range  and  ft  «  -dR/dt  radial  velocity, 

ft  ■  V  COS  9 

dft/dt  ■  -v  sin  e 

at 

d2ft/dt2  •  -v{$1n  9  +  cos  e  (de/dt)2} 

But  ■  °»  since  Is  constant. 
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■  -v  cos  e  (de/dt)2 


Dividing  both  sides  by  ft  ■  v  cos  8, 

-  -(dfl/dt)2  (5) 

R 

This  quantity  Is  called  the  "acc'rate  ratio/  and  the  estimate  of  It  Is 
designated  by  the  symbol  q.  ("Acc'rate"  refers  to  the  derivative  d2ft/dt2.) 

It  can  be  observed  that,  under  the  stated  conditions,  the  "acc'rate  ratio" 

(1)  Is  constant 

(2)  Is  proportional  to  the  square  of  the  aircraft's  turn  rate 

(3)  Is  Independent  of  the  direction  in  which  the  aircraft  Is  turning 

(4)  cannot  take  on  positive  values 

In  any  computer  solution,  derivatives  are  estimated  by  a  process  of  dif¬ 
ferencing.  Let  u0,  ult  u2,  u3,...be  values  of  a  function  equally  spaced  In 
time.  If  the  unit  of  time  be  taken  as  the  Increment  between  two  successive 
arguments,  then  u«  -  2ut  +  u2  becomes  an  estimate  of  the  second  derivative  at 
the  time  of  ur  it  does  not  matter  In  which  direction  time  Is  Increaslnq,  the 
value  of  the  estimate  remains  the  same. 

For  simplicity  and  clarity,  the  following  conventions  are  adopted: 

The  period  of  one  radar  antenna  rotation  (e.g.,  three  seconds)  Is  taken  as 
the  unit  of  time. 

Subscripts  are  expressed  In  multiples  of  this  time  unit  and  denote  "age." 
Thus  R0  Is  present  value  of  range,  R6  was  the  range  value  6  seconds  ago. 

The  subscript  "p"  (for  "predicted")  Is  used  In  place  of  the  subscript 
"-3." 

ft  Is  used  for  radial  velocity  and  Is  taken  In  a  positive  sense  for 
approaching  aircraft.  Thus  ft  ■  -dR/dt. 

Applying  the  foregoing,  It  Is  found  that 

^6  “  ^3  +  ^0  " 
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which  equation  yields  a  point  estimate  of  q3.  But  since  q  1$  presumed  con¬ 
stant,  the  subscript  can  be  dropped.  Having  now  an  estimate  of  q,  we  can 
advance  the  subscripts  and  write 

ft3  -  2ft  0  +  ftp  •  qft0 


Solving  for  ftp, 

ftp  -  (2  +  q)  ft0  -  ft3  (6) 

which  Is  simply  equation  (3)  In  slightly  altered  form. 

This  Is  the  basic  prediction  equation  and  Is  fundamental  to  the  algorithm. 
It  solves  the  problem  previously  posed,  and,  provided  turn-reversal  has  not 
just  occurred,  It  Is  used  for  range  computation. 

Now  at  an  FM  scan,  the  received  data  Is  equivalent,  not  to  ft,  but  to  the 
quantity  ft  •  hR,  where  h  Is  a  constant  whose  value  depends  upon  the  frequency 
modulation  rate.  Therefore,  there  exists  {at  each  FM  scan)  a  corollary 
requirement  to  predict  range  as  well  as  radial  velocity.  Since  range  predic¬ 
tion  Is  required  only  on  alternate  scans,  Integration  by  Simpson's  rule  Is 
well  suited  to  the  purpose.  Simply  stated, 

Rp  -  R3  +  +  ftp) 

Using  as  an  example  an  antenna  rotation  period  of  three  seconds,  (at  ■  3), 
and  substituting  the  derived  expression  for  ftp,  there  Is  achieved  the  remark¬ 
ably  simple  expression 


Rp  -  Ra  -  (6  +  q)  ft0  (7) 

Referring  to  the  prediction  equations  It  Is  apparent  that,  relative  to  the 
time  for  which  the  prediction  Is  made,  data  is  used  which  Is  up  to  two  scans 
"old"  (6  seconds  old  In  the  example  used).  During  this  time  frame,  should  the 
aircraft  markedly  alter  Its  maneuver  —  by  turn  reversal,  for  example  --  the 
data  may  be  so  greatly  perturbed  that  the  equations  cannot  be  used.  The  con¬ 
dition  Is  temporary,  lasting  only  until  the  aircraft  has  persisted  In  Its  new 
maneuver  for  a  period  of  two  scans  or  more,  but  must  be  Identified,  since  a 
momentary  change  In  procedure  Is  required. 

b.  ACQUIRING  THE  TARGET 

Suppose  a  CW  radar  with  an  FM  coefficient  h  »  0.002  Is  required  to  operate 
effectively  between  the  ranges  of  5  and  60  statute  miles.  This  operating 
range  Is  called  the  "Information  band,"  and  the  values  5  and  60  the  band 
edges.  All  Information  outside  this  band  Is  considered  suspect.  The 
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difference,  55  statute  miles,  Is  of  course,  the  band  width.  Converting  to 
metric  units,  the  band  edges  are  8,047m  and  96,561m,  respectively,  with  a  band 
width  of  88,514m.  Multiplying  by  h  ■  0.002  yields  the  “velocity  band,"  with 
band  edges  of  16.1  m/sec  and  193.1  m/sec.  If  the  difference  between  the  FM 
data  and  ft  does  not  fall  within  this  band,  the  point  estimate  of  the  range 
also  will  be  out-of-band.  Note  that  velocity  band  width  1$  177  m/sec. 

It  turns  out  that  If  a  suitable  value  of  q  Is  assumed  (q  >  -0.05,  say), 
track  can  be  Initiated  from  three  sets  of  data,  provided  they  are  not  all  of 
the  same  kind.  (If  there  Is  no  FM  data,  range  cannot  be  determined.  If  there 
Is  no  CM  data,  radial  velocity  cannot  be  determined,  which  In  turn  precludes 
range  determination.) 

U)  Track  Initiation  When  the  Data  Comes  From  Two  CW  Seans 
ahTTneTMTcar - - 

The  value  of  ft. Is  given  directly  by  the  CW  data,  but  Is  unknown  at  the  FM 
scan,  and  at  all  scans  where  data  Is  missing.  The  data  on  hand  will  be  desig¬ 
nated  CWj  and  FM j .  From  the  basic  properties  of  the  radar,  we  have 

FMj  -  ftj  -  hRj  (4) 

from  which 

«j  •  £  («J  ■  FMj)  (8) 


To  estimate  ftj,  a  value  of  q  Is  arbitrarily  assigned  (e.g . ,  q  » 
following  equations  are  written 

ftg  +  "  (2  +  q)  ftj 

ft9  +  ft3  »  (2  +  q)  ft6 

ft12  +  ftB  -  (2  +  q)  ft, 

etc. 


-0.05)  and  the 


until  a  system  of  n  simultaneous  equations  In  n  unknowns  1$  obtained,  which 
set  can  be  solved  for  ftj.  If  the  difference  ftj  -  FMj  falls  within  the  proper 

velocity  band,  track  Is  Initiated.  If  not,  the  oldest  data  Is  discarded  and 
another  attempt  Is  made  at  the  next  scan  with  suitable  data. 

EXAMPLE  1.  Available  data  Is  CW9,  CW3,  and  FM0. 

ft6  +  ft0  ■  (2  +  q)  CW3 

CW,  +  CW3  -  (2  +  q)  ft6 


356 


These  two  equations  In  two  unknowns  (ft6  and  A.)  can  be  solved  for  both 
unknowns.  However,  ft6  Is  not  needed  and  can  be  simply  eliminated,  leaving 

CW9  +  CW3  -  (2  +  q)  {(2  ♦  q)  CW3  -  A0} 

from  which 

&0  "  (2  ♦  q)  CW8  -  (9} 

and  of  course,  from  equation  (8), 

&0  "  ’jj’  (fto  ■  ™o) 


These  parameters  possess  dimension,  and  hence  proper  attention  must  be 
paid  to  scaling  In  the  computer. 

EXAMPLE  2.  Available  data  Is  CW6,  FM3,  and  CW0. 

CW6  +  CWq  -  (2  +  q)  Aa 

can  be  solved  for  the  only  unknown. 


and 


ft  -  i&JiiilB  (10) 

8  2  ♦  q 

«,  "  i  (ft,  -  PM,) 


Note  that  this  range  reduction  Is  valid  for  the  time  of  the  FM  data. 
Equation  (7)  now  yields  Rp  directly. 

Rp  -  R3  ■  (6  +  q)  CW0 

(2)  Track  Initiation  When  the  Data  Comes  From  One  CW  Scan 
and  Two  FM  Scans  “ 


Since  the  value  of  ft  Is  unknown  at  an  additional  FM  scan,  another  equation 
must  be  developed  to  complete  the  simultaneous  set.  This  Is  done  by  evalu¬ 
ating  the  definite  range  Integral  between  the  FM  scans,  utilizing  Simpson's 
rule.  Continuing  to  Illustrate  by  example,  suppose  the  available  data  to  be 
FMj,  CW3,  and  FM0.  By  Simpson's  rule 

Rq  ■  R6  -  ~  (ft|  +  4ftg  ♦  ft0) 

w 
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.£V.V5.,S  LVi  'i.l'i  Jar*.- A', 


Ly 


(11) 


If  At  *  3  sec,  this  simplifies  to 

Ro  *  R6  _  (l^g  4$3  +  Sq) 
Now  ft3  ■  CW3  and  applying  equation  (3)  yields 

$6  +  $0  ■  (2  ♦  q)  CW3 

hence 


From  equation  (4), 


and 


Rq  -  R6  -  (6  +  q)  CW3 
ftg  *•  hR6  ■  FMg 
fto  “  hR0  ■  FMq 


(12) 


Subtracting  and  transposing, 

h(Rfl  -  R6)  -  A0  ’  *6  “  PM0  +  FM6 

From  equation  (12) 

h(R0  -  R6)  -  -h(6  +  q)  CW3 


Consequently, 


ft0  -  *e  ■  FM0  -  FM6  -  h(6  +  q)  CW3  (13) 

the  sought  additional  equation  In  the  proper  unknowns.  Together  with  equation 
(11)  It  forms  the  required  simultaneous  pair.  Again,  R6  Is  not  needed  and  can 
be  eliminated,  this  time  by  simple  addition. 

2ft0  -  FM0  -  FM6  +  {2  +  q  -  h(6  +  q) }  CW3  (14) 


and 


Ro 


(3)  Further  Remarks  on  Track  Initiation 

If  It  could  be  known  that  an  aircraft  had  altered  Its  maneuver  during  the 
period  when  data  for  track  Initiation  was  being  collected,  It  would  become 
mandatory  to  delay  until  data  from  three  scans  reflecting  the  new  maneuver 
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became  available.  Probably  prospects  are  not  as  hopeless  as  they  may  at  first 
seem. 

If  the  first  attempted  range  computation  falls  out  of  band,  It  Is  assumed 
that  some  change  has  taken  place.  The  "oldest"  data  Is  discarded  and  the 
attempt  repeated  when  suitable  additional  information  Is  obtained.  In  this 
connection  It  Is  wise  to  keep  the  computational  band  width  as  narrow  as  prac¬ 
ticable.  Oust  because  the  radar  can  ''see"  a  target  Is  not  of  Itself  suffi¬ 
cient  reason  to  compute  Its  parameters. 

Consider  a  Doppler  radar  being  used  In  an  anti-aircraft  battery  configura¬ 
tion.  Ranges  so  great  that  the  missile  would  be  spent  before  reaching  the 
aircraft  need  not  be  Included  In  the  computational  band.  The  same  can  be  said 
for  ranges  so  small  that  Insufficient  time  to  permit  launching  remains. 

When  an  Isolated  case  of  FM  drop-out  occurs.  It  usually  Is  accompanied  by 
turn  reversal  within  less  than  two  scans  (whether  before  or  after  Is  unknown). 
Thus  In  the  sequence  CW9  •  miss  -  CW,  -  FM0,  the  data  at  CW9  Is  sometimes 
valid,  sometimes  suspect.  However,  if  CW9  »  CW,,  turn  reversal  Is  almost 
certainly  found  between  these  two  scans.  To  limit  the  application  of  this 
sequence  to  the  case  CW9  <  CW6  thus  Is  indicated. 

The  above  procedures  will  Identify  much  (but  not  all)  of  the  suspect 
data. 

The  three  examples  given  In  (1)  and  (2)  above  doubtless  represent  the  only 
practical  Initiation  sequences. 

c.  FOUR-SCAN  ACQUISITION 

It  Is  natural  to  Inquire  why  a  four-scan  acquisition  procedure  Is  not 
used,  since  the  additional  Information  would  make  It  possible  to  compute  the 
value  of  q,  rather  than  arbitrarily  assume  It. 

There  Is  no  theoretical  reason  why  »  four-scan  procedure  could  not  be 
employed.  There  are,  however,  two  sound,  practical  ones. 

(1)  Acquisition  will  certainly  be  delayed  one  scan,  merely  to  obtain  the 
additional  data. 

(2)  The  probability  that  four  successive  pieces  of  Information  will  not 
Include  a  maneuver  change  obviously  Is  less  than  the  probability  associated 
with  only  three. 

Four-scan  acquisition  Increases  both  the  risk  and  the  tlms  required.  In  a 
critical  situation,  the  price  may  be  prohibitively  high. 

There  can  be  no  objection  to  a  four-scan  supplemental  procedure,  provided 
the  additional  data  Is  compatible  (l.e.,  Indicates  no  perturbation  such  as 
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turn-reversal).  In  fact,  there  Is  one  Instance  Where  four-scan  may  be 
superior  to  three-scan.  We  discuss  that  case  forthwith. 

CASE  1.  CW12  -  miss  -  CW6  -  FM3  -  CW0. 

The  corresponding  three-scan  case  Is  the  least  accurate  of  the  procedures 
discussed  so  far,  due  to  the  fact  that  velocity  Is  estimated  by  extrapolation. 
Range  errors  can  be  quite  large,  and  tend  to  be  corrected  very  slowly  at 
first,  apparently  because  range  and  "q"  err  In  the  opposite  sense  (logically) 
with  respect  to  the  quantity  -  Ap. 

The  greatly  Increased  accuracy  of  the  four-scan  acquisition  procedure 
recommends  It.  The  mathematics  Is  relatively  simple.  We  have 

CW12  ♦  CW6  -  (2M)  ft9 

CW6  +  CW0  -  (2  +  q)  ft8 

Afl  +  A3  -  (2  +  q)  CW6 

Multiplying  the  last  equation  by  (2  +  q)  and  substituting  the  result  Into  the 
sum  of  the  first  two  equations, 

CWl2  +  2CW6  +  CW0  -  (2  +  q)2  CW6 


from  which 


+  CWn 


+2  ■  (2  +  q)s 


Essentially,  we  have  the  three-scan  case  of  paragraph  b(l),  Example  2,  except 
that  the  value  of  "q"  Is  computed  rather  than  assumed.  If,  however,  (2  +  q)2 
>  4,  turn-reversal  apparently  has  occurred.  In  this  event,  we  merely  discard 
CW12  and  revert  to  the  appropriate  three-scan  procedure. 

CASE  2.  CW9  -  KM6  -  CW3  -  FM0 

In  this  case  the  target  already  has  been  acquired  (else  CW9  would  have 
been  discarded).  The  four-scan  procedure  Is  tentatively  substituted  for  the 
usual  update,  provided  a  valid  value  of  "q"  Is  returned.  The  basic  equations 
are: 

L  .  cw»  *  c*a 

6  2  +  q 

CW3(2  +  q)  -  A6  +  A0 
h(R6  -  R0)  -  N«  “  FMs)  “  No  “  FMo) 


TT'^IWW-'PLTVi'tt1^ 


R6  -  R0  -  {6  +  q)  CW3  «  4CW3  +  (2  ♦  q)  CW3 

Solving  for  (2  +  q),  the  larger  root  of  the  equation 

(1  +  h)  CW3(2  +  q)2  +  (FM6  -  FM0  +  4hCW3)  (2  +  q)  -  2(CW9  +  CW3)  -  0 

1$  chosen.  If  (2  +  q)  >  2,  the  four- scan  procedure  Is  not  used.  Probably  a 
Class  2  fit*  has  occurred  and  the  range  should  be  "coasted"  (see  para  d(3)). 

If  (2  +  q)  <  2  (l.e.,  "q"  Is  negative),  we  proceed 


/  „  I  All  CWq  *  CW, 

•  (2  +  q)  CW3 - j  - 

«o  ■  £  (*,  -  f»o) 


CASE  3.  FM,  -  CU6  ■  FM,  -  CM, 

This  case  Is  strikingly  similar  to  the  preceding  one.  A  parallel  develop¬ 
ment  yields  the  following  quadratic  equation  In  (2  +  q). 

(1  -  h)  CW6(2  +  q)2  +  (FM,  -  FM,  -  4hCW6)  (2  +  q)  -  2(CW6  +  CW0)  -  0 

Notice  the  reversed  sign  of  the  terms  Involving  h. 


Continuing,  provided  (2  ♦  q)  < 

*3 


2. 

.  CW,  *  cw0 
2  +  q 


*3 


Up  ■  ^3  "  (6  +  q)  CWq 

Again,  If  (2  +  q)  >  2,  It  Is  likely  that  a  Class  2  fit  Is  present, 
d.  UPDATING  THE  RANGE  TRACK  FILE  SUBSEQUENT  TO  INITIATION 


For  a  maneuvering  aircraft,  the  radial  velocity  can  change  markedly  from 
scan  to  scan.  In  fact,  there  Is  some  justification  for  using  the  raw  Incoming 
data,  ft*,  without  smoothing.  However,  In  most  applications,  It  will  be  sus¬ 
pected  chat  the  ft^  data  contains  noise,  and  that  a  small  amount  of  smoothing 

will  be  beneficial . 


*See  section  III  for  definition  of  "Class  2  fit." 


361 


taatiflin 


LiJ  ■>— lf|,  ■ 


Letting  the  subscripts  "age"  (l.e.,  the  most  recent  track  file  value 
becomes  ft3),  the  basic  updating  equation  becomes 

R0  "  Rp  *  -  Rp)  (1) 

Radial  velocity  will  not  tend  to  seek  any  particular  value  with  passage  of 
time. 

We  have  seen  that  at  a  CW-scan 

■  CK 

However,  at  an  FM-scan,  the  best  we  can  do  Is  to  employ  equation  (4)  to 
estimate  Rm;  viz. 


Am  ■  FM  +  hRp  (15) 

Since,  In  the  two  cases,  the  value  of  Is  differently  arrived  at,  there  Is 

no  reason  to  suppose  that  the  coefficient  k  might  not  also  be  different. 
Therefore,  we  shall  use  k'  to  denote  the  velocity  filter  coefficient  at  a  CW- 
scan. 

The  smoothed  value  (ft0)  having  been  computed,  It  Is  employed  to  make  a  new 
velocity  prediction  for  use  at  the  following  scan.  From  equation  (3)  we 
obtain  Immediately 


ftp(new)  ■  (2  +  q) 

(1)  Range  Reduction  at  an  FM-Scan 

Range  Information  Is  available  only  at  FM  scans.  Provided  there  Is  suita¬ 
ble*  data,  equation  (8)  will  yield  a  point  estimate  of  the  range 

i  .  FM) 

It  Is  desirable  to  smooth  this  estimate  through  a  proper  filter. 

R0  ■  MI  -  ♦)  Up 


or  .qulv.lently 


Ro  "  Rp  +  *(Rm  “  Rp> 


(16) 


*How  to  determine  whether  or  not  the  data  Is  "suitable"  Is  the  subject  of  a 
later  section. 


^"ffrownm  TUEmr?!1'1 


For  a  smoothing  filter,  0  <  <l»  <  1,  of  course. 

Usually,  It  will  be  found  that  the  track  file  value  of  the  range  tends  to 
be  more  stable  than  the  point  estimate,  whereupon  a  fairly  small  value  of  i|>  Is 
Indicated. 

Equation  (16)  can  be  developed  Into  a  form  more  useful  for  computation  as 
follows:  From  equation  (15)  we  have 

K  ■  FM  ♦  hRp  (15) 

Subtracting  ftp  from  both  sides,  It  is  found 

-  Ap  ■  hRp  -  (rtp  *  fm) 

^m  "  ^p  *  ^Rp  *  ^^m 

from  which 


Substituting  this  expression  Into  equation  (16)  yields 

Ro  *  Rp  *  ||  (^m  *  ^p)  U?) 

which  computes  range  directly  from  the  basic  quantity  -  rtp. 


(2)  Range  Prediction  at  a  CW-Scan 

There  Is  no  range  information  at  a  CW-scan.  Range  Is  predicted  by  Inte¬ 
grating  between  FM  scans,  utilizing  Simpson's  rule.  This  Is  effected  by  equa¬ 
tion  (7),  repeated  here  for  convenience. 

Rp  -  R3  -  (6  +  q)  $0  (7) 

Since  Rp  and  R3  refer  to  FM  scans,  necessarily  Is  the  smoothed  velocity  at 
the  Intervening  CW-scan. 

(3)  "Coasting"  the  Range 

At  an  FM-scan,  when  there  Is  no  suitable  data,  equation  (17)  cannot  be 
used  to  update  the  range  track  file.  However,  there  Is  still  a  requirement  to 
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take  aircraft  movement  Into  account.  This  Is  done  by  a  process  variously 
known  as  "coasting"  or  "dead  reckoning." 

First,  It  Is  necessary  to  compute,  estimate,  or  arbitrarily  assign  a  value 
to  R0.  Then  the  range  can  be  "coasted"  by  Simpson's  rule.  In  expanded  form, 
after  "aging"  subscripts, 

Rg  •  R6  -  (ft6  +  4*3  +  ,A0) 

But  we  already  find  In  the  track  file  (still  employing  expanded  form) 

Rp  ■  R,  -  3*  (*,  +  4rt,  +  flp) 

Simple  subtraction  produces 


Rp  •  Rp  -  (ft,  -  «p)  (10) 

The  simplification  when  At  ■  3  seconds  Is  obvious, 
e.  UPDATING  THE  VALUE  OF  q 

Aircraft  maneuvers  are  subject  to  change.  Further,  an  assumed  value  of  q 
almost  certainly  Is  not  without  error.  It  Is  therefore  worthwhile  to  attempt 
to  measure  the  acc'rate  ratio  for  the  purpose  of  updating  the  value  of  q  car¬ 
ried  In  the  track  file. 

From  the  original  definition  of  q,  It  Is  seen  that  a  point  estimate  can  be 
obtained  from 


t  ■  ■** 

"8 

Multiplying  by  ft3  and  transposing 

(2  +  qm)  S3  -  ^6  "  ^m 

But  we  already  find  In  the  track  file  (after  "aging"  the  subscripts) 

(2  +  q)  ft3  -  ft6  -  rtp(old) 

Subtracting, 

-  0)  ft3  "  Am  -  Ap 


(19) 
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Since  q  Is  presumed  constant,  there  Is  no  need  for  a  prediction;  a  suitable 
filter  for  updating  is 


q(new)  -  q  +  5(qm  -  q) 

Substituting  equation  (19)  Into  this  expression  gives 

q(new)  -  q  +  4-  (ft„,  -  ftp)  (20) 

k3 

In  terms  of  the  basic  quantity  (ftffl  -  ftp).  The  track  file  value  of  q  Is  simply 
augmented  by  an  amount 

4~  (^m  ■  Rp) 

K3 

It  Is  known  that  q  cannot  take  on  positive  values.  Should  the  update 
produce  a  result  q(new)  >  0,  the  track  file  value  --  call  It  qTp  here  — 

should  be  reset  to  zero.  This  Is  easily  accomplished  by  passing  all  updates 
through  the  filter 

qTF  -  1  (q(new)  -  |q(new)|}  (21) 

The  process  of  differencing  in  the  computer  Is  Inherently  noisy.  The 
estimate  of  the  acc'rate  ratio,  being  based  on  a  second  order  difference, 

Is  doubly  noisy.  A  quite  small  value  of  5  therefore  Is  required  to  achieve 
sufficient  smoothing. 

f.  OPTIMUM  FILTER  COEFFICIENTS 

It  has  been  seen  that  the  updating  equations  for  range,  velocity,  and 
turn-rate  (q)  all  depend  upon  ftm  which,  after  all,  Is  the  sole  source  of  new 

Information.  In  each  case,  the  quantity  (ftm  -  ftp)  Is  multiplied  by  an 
appropriate  (k,  'll,  5)  weighting  coefficient.  Now  ftm  -  ftp  can  be  thought  of  as 
a  residual  or  as  an  error  but,  regardless  of  concept,  unarguably  contains 
"noise."  If  the  sum  of  the  weights  exceeds  unity,  l.e.,  If 

k  +  ^  +  5  >  1 

the  algorithm  will  act  as  a  noise  amplifier  and  the  solution  can  be  expected 
to  diverge.  This  results  In  the  following  limits  being  Imposed: 


k  +  t  +  5  <  1 


(22) 


I 

l 


Assuming  a  constant  value  of  q,  the  algorithm  equations  are: 

(1)  At  a  CW-scan 

K  -  cw 

ft0  -  k'  ft*,  +  (1  -  k')  rtp(old) 

Rp  ■  R3  -  (6  4  q)  ^0 

ftp(new)  -  (2  +  q)  ft0  -  ft3 

(2)  At  an  FM-scan 

ftm  *  FM  +  hRp 

ft0  -  kftm  +  (1  -  k)  ftp(old) 

R0  "  Rp  *  ^p(old) } 

ftp(new)  •  (2  +  q)  ft0  -  ft3 

Supposing  there  Is  an  error  e  in  the  data  at  a  CW-scan.  Then 
k'(ft|n  ♦  «)  ♦  (1  -  k' )  ftp  -  f*0  ♦  k'e 
R3  -  (6  +  q)  (ft0  +  k'e)  -  Rp  -  (6  +  q)  k'e 
and 

(2  +  q)  (ft0  +  k'e)  -  ft3  -  ftp(new)  ♦  (2  +  q)  k'e 

At  the  following  FM-snan  we  then  find 

FM  +  h{Rp  -  (6  +  q)  k'e}  »  ftm  -  h(6  +  q)  k'e 

Hh  -  h(6  +  q)  k'e}  4  (1  -  k)  {Rp  4  (2  +  q)  k'e} 

«  ft0  -  k{2  +•  q  +  h(6  4  q)}  k'e  4  (2  4  q)  k'e 
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(2  +  q)  $o  -  k( 2  +  q)  {2  +  q  +  h(6  +  q) }  k'e  +  (2  +  q)z  k'c  -  (ft3  +  k'e) 


■  Rp(new)  +  1(2  +  q)2  -  1}  k'e  -  k(2  +  q)  |2  +  q  +  h(6  +  q)}  k'e 


The  error  In  Rp(new)  VANISHES  NON-TRIVIALLY  when 

k(2  +  q)  {2  +  q  +  h{6  +  q)}  •  (2  +  q)2  -  1 

Thus  we  can  select  a  value  of  k,  the  FM-scan  velocity  filter  coefficient, 
which  prevents  an  error  In  the  CW  data  from  being  propagated  forward  into  the 
next  CW-scan.  Notice  that  k,  thus  selected,  Is  a  function  of  q  alone, 

A  form  mere  suitable  for  computation  is 


For  the  values  of  q  commonly  encountered,  k  usually  falls  between  0.69  and 
0.75.  Its  maximum  value  occurs  when  q  ■  0,  hen'te  Is  designated  k0.  From 
equation  (23) 

4k0(l  +  3h)  -  3  \ 

Equation  (22)  shows  that  both  <|»  and  6  must  be  small  In  comparison  with  k. 
Any  computation  of  them  which  Involves  subtraction  1;hu$  will  tend  to  be  noisy. 
There  seems  to  be  little  case  for  other  than  a  fixed'' value  of  either  i|»  or  £. 
Having  established  this  point,  we  can  recover  from  equation  (22)  the 
restraint 


♦  ♦  5  <  1  -  k0 

Dropping  the  Inequality  sign,  let  us  adopt  as  one  of  the  two  defining  equa¬ 
tions  for  K  and  <|i  the  expression 

♦  ♦  C  -  1  -  k0 

From  the  earlier  expression 

q(new)  -  q  +  4-  (ftm  -  *p)  (20) 

r3 

It  can  be  seen  that  for  very  high  speed  aircraft,  the  update  of  q  will  become 
too  sluggish  unless  $  Is  Increased.  A  procedure  such  as  the  following  Is 
suggested : 

Let  vmax  denote  the  maximum  expected  airspeed  (or  possibly  that  airspeed 
most  likely  to  be  encountered)  stated  In  m.sec-1.  Then 


367 


1  .  Mi. 

¥  450 

IhrlSSoi;ri1™?tS.ne°"usi;?,1",n3  e,U4t,°"  f0r  5  ,nd  The  *»  «pr«st»ns 

otaiUKiras  imZ  ?!  m  th1s  p,p,r> 

He  fiii5iLeve2L-liat  th!  err°r  ®  occurs  at  an  FM-scan,  an  analogous  procedure 
pm  c^]n0Wfn‘•/^!l^^?Aer^  s  nSS j^oth ,r*n9e  ®r1c*  radial  velocity  are  computed  at  an 
FM-scan,  an  additional  coefficient  (\|>)  appears  and  the  error  expression 

dJSTSZt!  eo*pl,e*ted  (^dMS  th«  109?c>.  ProcMdlSS  with  thS 

FM  +  e  +  hRp  ■  ftm  +  e 


Mjrf  +  «)  +  (1  -  k)  ftp(old)  -  ft0  +  ke 


(k  Is  now  a  known  function  of  q.) 

RP  ’  f  +  e  -  ^p(old)}  -  r„  -  |£ 

(2  +  q)  (ft0  +  ke )  -  ft3  ■  ftp(new)  +  (2  +  q)  ke 
At  the  following  CW-scan  we  find 

k‘ftm  +  (1  -  k1)  +  (2  +  q)  ke}  -  ft0  +  (1  -  k* )  (2  +  q)  ke 

r3  >  f  -  (6  +  q)  {ft0  +  (1  -  k')  (2  +  q)  ke} 

“  Rp  -  ■jf1  "  (6  +  q)  (1  -  k')  (2  +  q)  ke 

(2  +  q)  ft„  +  (2  +  q)2  (1  -  k* )  ke  -  (ft3  +  ke) 

•  Sp(new)  +  {(2  +  q)2  (l  .  k' )  -  1}  ke 

5i!tefFSih«S!i!d1?Kl02J,,.rfn9e  ?Hd  rad1a1  ve1°c1ty.  will  be  employed  at  the 
next  (FM)  scan,  the  equations  there  must  be  Investigated.  Continuing, 

FM  +  h{Rp  -  (6  +  q)  (1  -  k')  (2  +  q)  ke}  -  *c 


flm  *  -  h<6  +  d)  (1  -  k1)  (2  +  q)  ke 
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The  quantity  (ftm  -  ftp)  Is  used  to  update  all  parameters  In  the  track  file. 

Since  at  an  FM-scan  both  ftm  and  ftp  are  computed  values,  It  could  happen  that 
(ftm  -  ftp)  Is  without  error,  even  though  there  are  errors  In  the  track  file. 

In  this  event,  the  track  file  errors  remain  uncorrected.  The  filter  coeffi¬ 
cients  which  produce  this  effect  (no  correction)  can  be  taken  as  limiting 
values,  beyond  which  the  algorithm  diverges.  Obviously,  (ftm  -  ftp)  contains  no 

error  when 

-**  -  h(6  +  q)  (1  -  k')  (2  +  q)  ke  -  {(2  +  q)2  (1  -  k')  -  1}  ke 

Dividing  by  ke  and  transposing, 

1  -  J  “  (2  ♦  q)  (1  -  k')  (2  +  q  ♦  h(6  +  q)}  (24) 

Multiplying  by  k  and  substituting  from  equation  (23), 

k  -  *  -  (1  -  k')  {(2  +  q)2  -  1} 

This  expression  gives  an  upper  limit  for  k*.  For  5  ■  0.1  and  h  ■  0.002  its 
value  falls  between  0.76  and  0.81.  The  restriction  k 1  <  1  -  C  also  applies, 
of  course. 

Another  (lower)  limiting  value  for  k'  can  be  found  by  supposing  the  track 
file  range  error  to  be  completely  corrected  at  each  FM-scan.  Values  of  k' 
below  this  limit  will  produce  a  range  track  file  which  tends  to  be  both  oscil¬ 
latory  and  divergent.  The  development  proceeds 

Rp  .11.  (6  +  q)  (1  -  k')  (2  +  q)  ke  -  4  {ftffl  -  * 

-  h(6  +  q)  (1  -  k')  (2  +  q)  kc}  +  1  (ftp  +  [(2  +  q)2  (1  -  k')  -  1]  ke} 

-  R0  ’  (1  -  *)  fr  ■  (1  -  ♦)  (6  +  «l)  (1  -  k')  (2  +  q)  ke 

n 

+  £  1(2  +  q)2  (1  -  k')  -  1}  k« 

The  error  In  R0  vanishes  when 

(1  -♦){*+  (6  +  q)  (1  -  k')  (2  +  q)  hk}  -  {(1  -  k')  (2  +  q)2  -  1}  4* 
Substituting  from  equation  (23)  and  rearranging 

♦(1  -  *  +  k)  -  (1  -  k1)  {(1  -  *)  -  (1  -  *  -  k)  (2  +  q)2}  (25) 


from  which  the  lower  limit  of  k'  can  be  computed.  For  £  »  0.1  and  h  ■  0.002, 
the  value  falls  between  0.30  and  0.45. 

A  value  of  k'  about  midway  between  the  above  limits  should  achieve  near¬ 
optimum  smooth  range  tracking.  Therefore  let  us  require  that  the  error  In  RQ 
be  roughly  naif  that  In  R6  (l.e.,  at  the  previous  FM-scan).  Immediately,  we 
can  write 

"  -(1  -  *)  {♦  ♦  (6  +  q)  (1  -  k')  (2  +  q)  hk} 

+  1(1  -  k')  (2  +  q;2  -  1}  i|ik 

A  development  exactly  similar  to  the  preceding  one  yields 

♦(^  -  ♦  +  k)  ■  (1  -  k')  {(I  -  f)  -  (1  -  *  -  k)  (2  +  q)2}  (26) 

The  value  of  k'  supplied  by  equation  (26)  should  be  near  optimum.  For  £  ■  0.1 
and  h  ■  0.002,  It  varies  between  0.53  and  0.62. 

III.  SELECTION.  CORRELATION.  ANO  CLASSIFICATION  OF  RANGE  AND  VELOCITY  DATA 

Up  to  this  point,  the'maln  purpose  has  been  to  develop  mathematical  formu¬ 
lae  for  acquiring  and  tracking  a  single  aircraft  In  a  turn  of  constant  rate 
(which  rate  may  be  zero).  It  may  happen  that  the  expected  sequence  of  events 
Is  altered  so  drastically  that  the  standard  formulae  cannot  be  employed  to 
treat  the  Incoming  data.  Whether  these  perturbations  are  apparent  or  real,  It 
Is  necessary  to  detect  their  occurrence,  so  that  alternate  steps  can  be 
taken. 

In  an  attempt  to  perform  this  detection,  a  trlchotomous  device  called 
"double-gating"*  Is  employed,  by  which  the  data  Is  evaluated  and  placed  Into 
one  of  three  categories,  as  follows: 

(1)  Class  1  fit.  The  Incoming  data  Is  consistent  with  the  predicted 
values,  and  thus  can  be  used  to  update  the  track  file  parameters. 

(2)  Class  2  fit.  The  Incoming  data  Is  Inconsistent  with  the  predicted 
values,  but  Is  not  Inconsistent  with  the  most  recent  track  file  values.  It  Is 
assumed  that  some  change  has  taken  place,  and  therefore  only  certain  of  the 
track  file  parameters  can  be  updated.  (For  example,  q  and  R  are  not  updated.) 
At  a  Class  2  fit,  k  »  k1  ■  0.5  probably  suffices. 


*In  this  paper,  "gate"  Is  synonymous  with  "window." 


(3)  Class  0  fit  -  a  miss.  The  Incoming  data  Is  either  erroneous  or  from 
an  extraneous  source,  and  hence  cannot  be  used.  This  class  Includes  the  case 
of  missing  data  (e.g.,  FM  drop-out). 

a.  TURN  REVERSAL 

Perhaps  the  data  are  perturbed  most  violently  when  a  high-speed  maneuver¬ 
ing  aircraft,  crossing  the  radar  line-of-slght  at  a  wide  angle,  suddenly 
reverses  Its  direction  of  turn.  In  fact,  a  suitable  criterion  for  setting  the 
"outer"  gate  Is  the  desired  maximum  detectable  perturbation  of  this  type. 

When  a  flight  path  Is  convex  (as  viewed  from  the  radar),  the  aircraft 
Is  turning  toward  a  receding  aspect,  and  thus  Is  an  Item  of  diminishing 
Interest.  Conversely,  when  the  flight  path  Is  concave  toward  the  radar,  the 
aircraft  Is  turning  toward  a  more  direct  approach.  When  turn  reversal  (at  a 
sufficiently  high  rate  of  turn)  has  produced  the  latter  state  of  affairs,  the 
data  at  the  following  scan  Invariably  will  exhibit  certain  symptoms.  They 
are: 


(1)  Acc'rate  will  appear  to  be  very  large  and  POSITIVE. 

(2)  The  absolute  value  of  the  difference  |ftm  -  ftp|  will  be  large  —  per¬ 
haps  even  beyond  the  outer  gate  (l.e.,  a  miss). 

(3)  When  the  predicted  velocity  Is  replaced  by  the  old  track  file  velo¬ 
city,  the  absolute  value  of  the  difference  always  Is  reduced.  In  other  words 

|ftm  -  ft3|  <  |ftm  -  ftp  I 

Moreover,  this  reduced  difference  rarely  falls  outside  the  window.  Thus,  the 
substitution  (of  ft3  for  Rp)  avoids  declaring  a  miss  with  good  data  present. 

(4)  Because  acc'rate  Is  based  upon  a  second  difference,  the  "large  and 
POSITIVE"  symptom  will  persist  for  an  additional  scan. 

(5)  So,  too,  will  persist  the  SIGN  of  the  residual  (ftm  -  ftp).  Thus,  for 
the  two  scans  following  turn  reversal 

(*„  '  »■)"«»  .  c 
-  ftp)°ld 

b.  TURBINE  RETURNS  AND  OTHER  LARGE  TRANSIENTS 

Sometimes  perturbations  In  the  data  are  not  accompanied  by  corresponding 
changes  In  the  flight  pattern  of  the  aircraft.  Such  a  case  might  occur  when 
the  aircraft  Is  approaching  head-on,  so  that  the  radar  "sees"  a  rotating  tur¬ 
bine  Instead  of  tne  aircraft  skin,  with  resulting  frequency  shift.  The 


minimum  expected  frequency  shift  of  this  type  Is  a  criterion  for  setting  the 
"inner"  gate. 


These  perturbations  rarely  are  persistent,  enabling  us  to  Identify  them 
merely  by  inspecting  the  data  at  the  following  scan.  Applying  the  yardstick 
of  the  preceding  subsection,  we  find 

(1)  The  point  estimate  of  the  acc'rate  ratio  will  be  very  large,  but  Is 
equally  likely  to  be  positive  or  negative. 

(2)  The  difference  (ftm  -  ftp)  usually  Is  large  --  but  can  be  of  either 
sign. 

(3)  Substitution  of  the  old  track  file  velocity  for  the  predicted  velo¬ 
city  does  not  necessarily  reduce  the  difference. 

(4)  At  the  following  scan,  the  point  estimate  of  the  acc'rate  ratio  will 
tend  to  be  large,  but  of  opposite  sign. 

(5)  The  residual  (ft-  -  ft.)  will  change  sign  at  the  following  scan.  In 
fact,  since  ftp(new)  presumably  Is  based  upon  erroneous  data,  whereas  ftp(old) 

Is  not,  we  expect 


(ftm  -  ftD)new  ^  _1 
(Am  "  &p)°1d 


C.  DATA  SELECTION 

The  correlation  process  begins  as  follows:  Does  the  measured  radial 
velocity  differ  from  both  the  predicted  value  and  the  latest  track  file  value 
by  more  than  the  outer  gate?  If  so,  a  miss  Is  declared.  An  exactly  similar 
test  Is  performed  upon  the  azimuth  measurement.  (NOTE:  Azimuth  gate  should 
vary  Inversely  with  range.) 

If  no  miss  Is  declared,  the  following  tests  are  performed: 

(1)  Does  the  measured  radial  velocity  differ  from  the  predicted  value  by 
less  than  the  Inner  gate? 

(2)  Is  the  point  estimate  of  q  less  than  or  equal  to  zero?  (To  accommo¬ 
date  noise  In  the  data,  small  positive  values  should  be  considered  as  zero.) 

If  the  answer  to  either  of  these  questions  Is  "no,"  a  Class  2  fit  Is 
declared.  But  If  both  questions  are  answered  "yes,"  the  result  Is  a  Class  1 
fit.  At  the  second  of  consecutive  Class  2  fits,  the  ratio 


p  . 
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Is  examined.  This  also  produces  three  categories: 

(1)  p  <  -1.  This  Implies  a  turbine  return  or  other  large  transient  at 
the  previous  scan.  Since  only  Rm  at  that  scan  Is  suspect,  substitution  of  the 
appropriate  Rp  for  R3  Is  Indicated.  Certain  derived  values  may  have  to  be 

recomputed . 

(2)  p  >  0.  This  Implies  a  turn  reversal  before  the  previous  scan.  As  a 
result,  several  actions  are  required: 

(a)  Do  not  update  the  acc'rate  ratio.  In  effect,  Is  missing. 

(b)  Do  not  update  the  range  at  the  FM-scan  —  dead  reckon  It  Instead 
employing  Simpson's  rule.  Effectively,  Rp  Is  missing,  leaving  nothing  upon 

which  to  base  a  range  reduction. 

(c)  Do  not  compute  the  radial  velocity.  The  required  Information  will  be 
found  to  lie  In  disparate  sets.  In  this  case,  however,  there  Is  sufficient 
Justification  for  substituting  the  appropriate  values  of  Rm  for  and  R0, 

respectively. 

(3)  -1  <  p  <  0.  No  clear  Implication.  Compute  the  radial  velocity  and 
coast  the  range.  Nothing  better  can  be  offered. 

d.  GATE  SIZE 

The  purpose  of  any  "gate"  or  "window"  Is  to  admit  all  truthful  evidence 
while  screening  out  all  that  Is  false.  Unfortunately,  there  Is  no  perfect 
gate  size.  A  gate  which  admits  all  the  desirable  data  may  also  admit  much 
that  Is  not  wanted.  And  the  converse  Is  true.  A  gate  which  screens  out  all 
erroneous  data  may  reduce  to  a  trickle  the  flow  of  useful  information. 

All  that  ever  can  be  said  Is  that  a  gate  accomplishes  each  part  of  Its 
two-fold  mission  to  some  statistical  probability.  Clearly,  such  probabilities 
depend  upon  not  only  equipment  design,  but  upon  the  use  to  which  the  equipment 
Is  put,  as  well  as  the  performance  characteristics  of  the  expected  targets. 
Lacking  specifics,  only  general  suggestions  can  be  made. 

(1)  The  Outer  Gate.  Obviously,  the  principal  purpose  of  the  outer  gate 
Is  to  screen  out  unwanted  returns  from  extraneous  sources.  It.  must,  however, 
be  large  enough  to  permit  detection  of  certain  specified  maneuvers.  For  exam¬ 
ple,  If  At  *  3  sec,  and  It  Is  desired  to  detect  turn-reversal  on  a  target 
undergoing  6  g's  acceleration  (58.8  m/sec/sec),  an  outer  gate  of  at  least 
176.4  m/sec  will  be  required.  See  also  paragraph  (3)  below. 

(2)  The  Azimuth  Gate  Is  used  In  conjunction  with  the  outer  gate,  and  for 
the  same  primary  purpose.  It  must  be  (In  radians)  at  least  vAt/R.  After  a 
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miss,  both  the  outer  gate  and  azimuth  gate  should  be  Increased,  though  perhaps 
not  doubled.  A  multiplier  of  about  1.6  or  1.7  Is  suggested. 

(3)  The  Inner  Gate  Is  NOT  used  primarily  to  eliminate  unwanted  data,  but 
rather  to  identify  perturbations  in  the  desired  data,  so  that  some  special 
action  can  be  taken.  To  cite  another  example,  suppose  the  2a  noise  level  to 
be  25  m/sec.  Further  suppose  that  a  turbine  return  causes  a  perturbation  of 
110  m/sec.  The  Inner  gate  should  be  set  to  some  value  between  25  and  85 
m/sec.  If  v  denotes  noise  level  and  t  the  turbine  or  transient,  the  somewhat 
arbitrary  formula 

(  V  +  T )  V 

Inner  gate  ■  v  +  ttttt . - . 

2(450  ♦  vj 

yields  a  satisfactory  value.  The  outer  gate  should  be  at  least  double  the 
inner  gate. 

(4)  The  Q-Gate.  In  paragraph  IIIc(2)  above,  It  was  established  that  a 
Class  2  fit  snou'IcTbe  declared  whenever  the  point  estimate  of  q  exceeds  a 
certain  positive  value.  This  effectively  creates  another  window.  Lest  It 
superficially  appear  that  the  q-gate  duplicates  the  function  of  the  Inner  gate 
and  therefore  Is  redundant,  let  us  point  out  two  essential  differences. 

First,  the  Q-gate  Is  one-sided,  leaving  negative  values  undisturbed.  Second, 
the  Q-gate  Is  most  sensitive  to  aircraft  of  low  radial  velocity.  From  equa¬ 
tion  (19),  It  Is  seen  that 


Am  -  ftn 

qm  "  q +  -B 


The  presence  of  ft,  In  the  denominator  Is  responsible  for  the  essential  dif¬ 
ference  between  the  Q-gate  and  the  Inner  gate.  The  q-gate  is  quite  effective 
against  aircraft  which,  crossing  at  a  wide  angle,  begin  a  turn  toward  the 
radar  site.  A  word  of  extreme  caution  Is  necessary  however. 

At  successive  scans,  the  roles  of  CW  and  FM  Information  are  Interchanged. 
If  the  track  file  value  of  q  Is  greatly  In  error,  It  will  cause  the  value  of 
qm,  the  point  estimate,  to  oscillate.  If  the  Q-gate  Is  too  small,  q(1|  might 

fall  alternately  In  and  out  of  the  window,  with  catastrophic  result.  No 
appreciably  large  probability  of  this  occurrence  can  be  tolerated,  since  the 
condition  can  arise  If  a  maneuvering  aircraft  abandons  Its  maneuver  for  a 
direct  attack  upon  the  radar  site  Itself. 

A  Q-gate  of  0.15  to  0.2  Is  suggested. 

IV.  THE  AZIMUTH  PROBLEM 
a.  MEASUREMENT  AND  PREDICTION  OF  A7IMUTH 

In  general,  CVI  radars  can  determine  radial  velocity  by  a  direct  phase 
comparison  In  the  receiver,  but  can  determine  azimuth  only  by  locating  the 
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region  of  maximum  signal  strength,  or  by  detecting  when  that  signal  strength 
exceeds  a  certain  threshold. 

Evidently,  then,  the  measurement  of  azimuth  Is  less  precise  (l.e..  Is 
"noisier")  than  that  of  radial  velocity.  To  predict  azimuth  from  track  file 
history  alone  is  to  be  subject  to  occasional  extrapolation  errors  too  large  to 
be  tolerated.  In  an  attempt  to  control  this,  two  devices  are  used.  It  has 
been  seen  that 


A  ■  v  cos  e 
and 

dft/dt  ■  -v  sin  e  ~ 
at 

Letting  $  denote  the  azimuth  and  noting  that  under  the  assumed  conditions  a 
sufficiently  close  approximation  to  R  4^  Is  given  by 


R  di  ■  v  sin  e 


then 

dft/dt  >  -  R 

'  dt  dt 

from  which 

ds/dt  ■  •  A  •  dft/dt 
'  R  de/dt 

which  allows  azimuth  prediction  to  be  based  upon  the  more  accurately  known 
parameters  R,  R,  and  /^q.  The  sign  of  de/dt  Is  as  yet  unknown,  but  by 
restricting  use  of  the  formula  to  the  case  of  a  Class  1  fit,  It  Insures  that 
the  sign  will  not  have  changed  during  the  last  two  scans,  and  hence,  can  be 
determined  by  comparison  of  recent  changes  In  R  and  $.  If  e0  ■  03,  simply  set 
Op  -  e0  *  *3* 

If  a  Class  1  fit  does  not  exist,  or  If  the  sign  of  /^q  cannot  be  determ¬ 
ined  (e.g.,  when  Ro  -  *3  ■  0),  or  If  q  Is,  numerically,  very  small  (q  >  -0.01, 
say),  azimuth  must  be  extrapolated.  The  formula 

Bp  ■  + 

Is  recommended.  If  the  coefficient  u  Is  assigned  a  value  less  than  unity,  the 
cumulative  azimuth  correction  (In  the  case  of  several  Iterations)  will  be 
limited,  thereby  preventing  "runaway." 
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b.  AZIMUTH  CORRELATION 


At  track  initiation,  a  fixed  azimuth  "window"  must  be  used,  since  range  Is 
not  known.  For  subsonic  velocities,  a  window  of  0.012  to  0.020  radians  per 
second  appears  suitable.  A  fixed  window  tends  to  Inhibit  track  initiation  on 
laterally  flying  targets  at  short  ranges,  and  therefore  its  value  should  be 
chosen  according  to  the  desires  In  that  area. 

After  Initiation,  a  window  of  the  form  a  +  b/R  Is  usrd,  where  the  first 
term  accommodates  the  noise,  the  second  term  the  range.  If  a  Is  expressed  In 
radians  per  second,  then  b  should  be  expressed  In  meters  per  second,  R  In 
meters.  The  variable  window  should  be  equal  to  the  fixed  window  at  some 
selected  midrange. 

The  Incoming  data,  6m,  Is  compared  both  to  ep  and  to  B3  (subscript 

"aged").  If  both  differences  exceed  the  window,  a  "miss"  Is  declared.  If 
neither  azImutOata  nor  radial  velocity  data  produces  a  miss,  azimuth  Is 
updated  by 


+  (1  -  n) 

where  ex  Is  either  Bp  or  e3,  whichever  lies  closer  to  The  coefficient  n 

probably  should  be  chosen  between  0.5  and  0.8  In  order  to  given  some  weight  to 
the  track  file  data.  (Some  weight  already  had  been  given  by  the  selection 
of  »„.) 

In  the  event  of  a  miss,  simply  set  80  ■  9p(old). 

V.  ERRORS  INDUCED  BY  THE  POLAR  COORDINATE  SYSTEM 


So  far,  the  development  has  assumed  that  azimuth  changes  produce  negligi¬ 
bly  small  errors.  Strictly  speaking,  that  Is  not  always  true.  Let  us  examine 
Figure  1,  following. 

Let  an  aircraft  fly  a  curved  flight  path  (center  of  curvature  at  0)  from  A 
to  B  at  speed  v  during  the  period  of  one  scan.  Let  9  be  a  measure  of  aircraft 
heading,  referenced  to  south.  For  a  radar  positioned  at  C,  let  a  be  a  measure 
of  aircraft  azimuth,  referenced  to  north.  With  these  conventions.  0  +  o  Is 
the  aspect  angle  and  v  cos  (9  +  a)  ■  R  gives  the  radial  velocity  (positive  for 
approaching  aircraft). 

As  the  aircraft  moves  from  A  to  B,  the  flight  path  Is  concave  toward  the 
radar,  and  the  aspect  angle  changes  more  slowly  than  does  the  aircraft  head¬ 
ing.  Thus  the  apparent  turn  rate  Is  less  than  the  actual,  and  there  is  an 
apparent  outward  acceleration.  Should  the  flight  path  be  convex  toward  the 
radar,  the  aspect  angle  changes  more  rapidly  than  does  the  aircraft  heading, 
so  that  the  apparent  turn  rate  is  greater  than  the  actual.  But  the  apparent 
acceleration  Is  still  outward.  This  apparent  outward  acceleration  Is  a  func¬ 
tion  of  range,  turn  rate,  aircraft  speed,  and  aspect  angle.  It  typically 
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appears  In  the  equations  of  motion  In  a  polar  coordinate  system.  In  so  far  as 
It  arises  solely  because  of  the  coordinate  system  being  used,  It  Is  reminis¬ 
cent  of  Coriolis'  acceleration. 

In  designing  a  Doppler  radar  of  the  type  with  which  we  are  here  concerned, 
It  Is  arbitrarily  decided  to  observe  only  approaching  aircraft  whose  radial 
velocity  exceeds  a  specified  threshold.  Were  the  aircraft  to  fly  a  complete 
circle,  the  effect  can  be  likened  to  observing  the  sun  during  a  24-hour 
period.  The  aircraft  "rises"  out  of  the  band-stop  filter,  reaches  maximum 
radial  velocity  at  "noon,"  after  which  the  radial  velocity  decreases  at  a 
faster  and  faster  rate  until  It  disappears  behind  the  band-stop  filter. 

When  the  aircraft  "rises,"  the  square  of  the  turn  rate  appears  to  be  too 
small.  As  the  aircraft  "sets,"  the  square  of  the  turn  rate  appears  to  be  too 
large.  For  the  observable  sector,  then,  the  square  of  the  apparent  turn  rate 
Increases  monofonlcaTly. 

Now  the  observed  value  of  q,  the  acc'rate  ratio,  being  computed  from 
actual  observations,  would  seem  to  be  the  proper  parameter  for  predicting 
future  observations  (of  radial  velocity)  —  and  so  It  Is.  However,  from  the 
original  definition  of  q 


'  ^n  "  +  ^n-2 

J "  Vl 
Rn-1 

It  Is  seen  that  the  estimate  of  q,  being  based  upon  the  second  difference,  Is 
valid  only  at  the  previous  scan  —  and  the  value  of  q  Is  known  to  be  changing 
monotonlcally. 

The  trouble  with  the  estimate  of  q  Is  not  that  It  Is  In  error,  but  that  It 
Is  too  "old."  As  a  result,  the  prediction  multiplier  (2  +  q)  always  will  be 
too  large,  as  will  Rp  and  track  file  range.  Under  extreme  conditions,  this 

"over"  range  bias  can  exceed  tolerances.  A  more  up-to-date  estimate  of  q  Is 
needed.  Let  us  call  the  required  estimate  qp. 

The  basic  updating  equation  Is 

q(new)  -  q  +  -  Ap) 

and  should  be  used  In  normal  fashion.  Now  It  Is  known  that  q  will  decrease 
monotonlcally  as  long  as  the  present  aircraft  maneuver  persists.  Hence,  the 
corrective  equation 
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suggests  Itself,  with  the  proviso  that 

(ftm  “  ftp)  <  0 
Ko 

This  latter  Is  easily  accomplished  by  employing  the  filter 
qp  -  q(new)  Ap)  .  |ftm  -  ftpl} 

The  value  of  qD  Is  not  carried  In  the  track  file.  It  Is  used  to  predict 
Rp(new)  and,  at  CW-scans,  Rp,  then  discarded. 

When  qp  Is  thus  used  for  prediction,  an  unexpected  dividend  accrues. 
Repeating  for  convenience 

ftp(new)  •  (2  *♦*  q)  ft0  -  ft3  (6) 

and 

Rp  •  -(6  +  q)  ft0  ♦  R3  (7) 

It  Is  seen  that  when  qp  Is  used  for  q,  ft0  can  be  cancelled  out  of  the  correc¬ 
tion  term,  leaving 

ftp(new)  -  (2  +  qnew)  ft0  -  ft3  +  C(Am  -  ftp) 
and 

Rp  *  "  (®  +  ^new)  “  *(^m  *  ^p) 

still  subject  to  the  proviso  that  (ftm  -  Ap)  <  0. 
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EXAMINATION  OF  SIZE  EFFECTS  IN  THE 
FAILURE  PREDICTION  OF  CERAMIC  MATERIAL 


D.  M.  Neal  and  E.  M.  Lenoe 

Army  Materials  and  Mechanics  Research  Center 
Watertown,  Massachusetts  02172 


ABSTRACT 

Probability  of  structural  failure  of  a  large  hydro-burst  ceramic  Slip 
Cast  Fused  Silica  (S.C.F.S.)  ring  was  predicted  from  a  data  base  of  four 
point  bend  specimens.  Both  statistical  surface  and  volumetric  flaw  distri¬ 
bution  theory  were  considered.  The  major  contributions  of  this  paper  are 
determination  of  an  acceptable  size  effect  relationship  in  predicting 
failure  of  S.C.F.S,  and  the  recognition  that  uncontaminated  test  data  is 
necessary  in  the  prediction  process.  The  removal  of  outliers  and  multi- 
modality  from  the  test  data  will  define  the  uncontaminated  data.  There  was 
excellent  agreement  (less  than  11%  error)  between  predicted  and  actual 
experimental  results  when  surface  flaw  theory  is  applied  to  uncontaminated 
data.  The  mini-max  principle  in  conjunction  with  the  maximum  likelihood 
(M.L.)  method  i3  used  to  determine  the  outliers.  The  Quantile  Box  Plot 
method  is  applied  when  examining  for  multi-modality  and  outliers. 

By  considering  only  non- contaminated  data,  the  essential  need  for 
equality  of  coefficient  of  variations  between  specimen  test  data  and  the 
structural  component  materials  is  satisfied  in  the  predictive  process. 
Removing  outliers  and  multi-modality  (data  values  resulting  from  errors  in 
manufacturing  or  testing)  insures  the  same  basic  properties  of  the  ring  and 
flexure  specimen  materials.  Therefore,  application  of  acceptable  size 
effects  relations  in  the  prediction  process  can  be  more  successful  if  types 
and  distributions  of  flaws  are  similar  in  both  components  and  only  size 
governs  differences  in  their  mean  strengths. 
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INTRODUCTION 


Ceramic  materials  often  exhibit  a  size  effect  relationship  with 
respect  to  failure;  that  is  a  small  test  specimen  will  fail  at  higher 
stress  levels  than  a  larger  one  subjected  to  similar  loading.  The  rationale: 
since  the  strength  of  a  component  is  governed  by  chance  that  a  severe 
stress  concentration  (c)  will  be  subjected  to  a  stress  (s)  such  that  the 
local  stress  (sc)  exceeds  the  material  strength  and  the  chances  of  a  more 
severe  c  value  exists  in  the  larger  component  thereby  resulting  in  a  lower 
failure  stress.  Since  ceramic  materials  tested  at  ambiont  temperatures  do 
not  deform  plastically  and  relieve  these  stresses,  provides  a  reasonable 
verification  for  the  above  argument.  It  should  also  be  noted  that  relatively 
low  strength  values  could  result  if  no  severe  flaws  coincide  with  the 
maximum  stress  and  failure  occurs  at  a  severe  flaw  subjected  to  a  lower 
stress  at  a  position  where  (sc)  is  maximum. 

The  primary  objective  of  this  paper  is  to  determine  a  desirable  method 
for  predicting  the  mean  failure  stress  of  a  relatively  large  ring  from  the 
test  results  of  a  small  size  flexure  specimen  (see  Figure  1),  where  both 
materials  are  made  from  identical  S.C.F.S  material.  The  ring  and  flexure 
specimens  were  obtained  from  missile  radomes.  The  rings  were  subjected  to 
hydroburst  tests  and  the  flexure  bars  were  tested  as  shown  in  Figure  1. 

The  tests  were  conducted  in  order  to  establish  quality  control  of  the 
radome.  If  an  acceptable  size  effect  relationship  can  be  established 
between  the  two  components  then  the  need  for  continued  relatively  expensive 
test  of  rings  could  be  eliminated.  More  importantly  a  possible  failure 
prediction  methodology  for  S.C.F.S.  as  related  to  size  effects  will  be  made 
available.  The  authors  have  been  fortunate  in  that  considerable  amount  of 
data  has  been  made  available  by  the  Raytheon  Co.  of  Bedford,  Mass.  The 
data  was  separated  into  eight,  billets  containing  both  ring  and  flexure  test 
results  providing  combined  total  of  1300  specimens. 

The  authors  examined  the  merits  of  using  volume  versus  surface rf law 
theory  in  the  failure  prediction  process.  The  conventional  Weibull111 
method  was  applied  in  the  failure  prediction  process  with  some  success  if  the 
maximum  stressed  regions  are  considered.  Another  procedure2  using  a  surface 
flaw  distribution  theory  was  also  applied.  With  the  sample  sizes  available, 
the  opportunities  existed  for  the  authors  to  systematically  establish  an 
acceptable  size  dependent  failure  prediction  method  for  S.C.F.S.,  with  a 
reasonable  degree  of  certainty. 

In  order  to  eliminate  both  parasitic  stress2  (producing  low  tensile 
stresses)  and  high  tensile  stresses  (resulting  from  failures  remote  from 
the  maximum  stressed  region),  both  robust  estimating  procedure3  and  the 
Quantile  Box  Plot  was  applied  to  the  data.  These  procedures  will  recognize 
the  outliers  and  establish  uni-modal  distributions  in  formal  manner.  Appli¬ 
cation  of  these  procedures  will  result  in  data  that  represents  the  essence 
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of  the  material  strength  capabilities  in  addition  to  providing  a  more 
acceptable  representation  of  the  data.  This  is  particularly  important  when 
attempting  to  represent  flexure  test  results  adequately  since  this  size 
specimen  is  often  vulnerable  to  manufacturing  or  testing  errors. 

Robustness  Method 

The  outliers  are  determined  in  a  formal  manner  by  applying  a  robust 
method  involving  application  of  the  maximum  likelihood  (M. L.)  estimation 
where  the  residuals  are  weighted  in  a  systematic  manner.  The  computed 
weights  describe  the  relative  importance  of  the  data  points.  For  example, 
a  zero  weight  should  indicate  exclusion  of  a  point.  It  should  be  emphasized 
that  removing  outliers  without  a  valid  reason  is  poor  practice.  Outliers 
should  be  examined  for  errors  in  testing  or  possible  material  defects.  The 
removal  of  outliers  (bad  data)  will  essentially  define  robust  data,  The 
robust  procedures  applied  in  this  paper  involves  using  both  the  M-estimating 
technique  of  Huber*’  and  Andrews0.  Initially  the  Huber  technique  is  applied 
in  order  to  determine  a  robust  location  parameter  (weighted  mean) .  The 
Andrew's  function  is  then  applied  using  location  parameter  estimated  from 
the  Huber  result.  It  should  be  noted  that  this  robust  method  requires  a 
uni-modal  distribution  of  the  data,  therefore  initial  application  of  the 
Quantile  Bbx  Plot  should  be  made. 

The  Hubei*  m- estimation  technique  which  involves  defining  the  likelihood 
function 


k(^)  ■  -rj-f(X^-e),  "•»<•<•»  (1) 

i-1 


where  f  is  a  contaminated  normal  distribution, 

X^  ■  data,' 

0  ■  location  parameter  and 
N  ■  sample  size 
By  maximizing  log  L(e)  such  that 

Z>KXr«)  -  0,  (2) 

where  ^  ■  f'/f 

then  the  Solution  of  (2)  is  M.L.  estimate  of  e  designated  as$.  In  order  to 
represent  <Ji  in  scale  invariant  form,  equation  (2)  can  be  rewritten  as 
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Z*((^)  .  0  (3) 

with  d  equal  to  the  estimate  of  scale.  The  scale  is  often  defined  as 
d  ■  median! -  median  (X^) |  /.674S 

or  simple  M.A.D./.6745  (4) 

This  estimate  is  considerably  more  robust  than  using  the  complete  samples 
which  could  result  in  poor  representation  of  the  actual  scale. 

By  solving 


W^-e)  -  0 


(5) 


where 


V  -  jr  lr>  *  ci 

jcj  sign  (r)  |ri  >  c1  , 

Cj  is  defined  as  the  tuning  constant  and 


An  iterative  process  is  then  used  in  the  solution  of  (5)  such  that 
when  the  differences  in  W.  become  negligible  provides  the  necessary  criteria 
for  an  acceptable  solution  for  the  a  and  W.  values.  For  1.345  the 
Huber’s  ♦  function  provides  a  95%  efficiency  .  1 

With  estimate  of  d  determined  from  the  solution  of  (2)  the  Iteration 
is  continued  where  the  ¥  function  is  now  defined  as 


-ffT'1;  1?!  z  £*  m 

This  new  function  is  called  the  Andrew's  wave  equation.  In  order  to  obtain 
the  desired  robust  data  for  the  function,  the  tuning  was  adjusted  to 
c^l.34  and  the  scale  defined  as  in  equation  (4). 
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It  should  be  noted  that  Andrew's  function  was  selected  for  its  ability  to 
describe  outliers  as  data  with  essentially  zero  weights. 

Quantile  Box  Plot 

A  general  description  of  the  Quantile  Box  Plot  is  shown  in  Figure  2. 
Where  the  quantile  function  is  defined  as 

Q  (u)  ■  F'1  (u),  0  t  u  i  1  (7) 

that  is,  if  the  random  variable  x  with  distribution  function  given  by 
F(x),  then  the  root  of  F(x)  ■  u,  0  *  u  s  1  is  the  p  quantile  of  F(x). 
From  the  ordered  statistic  x,  is  x-s  .  .  .  ,  x  ,  Q  is  defined  as  piece 
wise  linear  function  with  interval  (0,1)  divided  into  2n  subintervals. 
Therefore  representing  Q  as 


Q (  2n  )  “  xj  »  j  ■  1,  2,  . . .  ,  n.  (8) 

In  order  to  interpolate 


where  n  equals  the  sample  size  . 

The  box  boundaries  are  defined  as 
Q  (.25)  to  Q  (.75) 

Q  (.125)  to  Q  (.875) 

Q  (.0625)  to  Q  (.9375) 

The  Quantile  function  Q(u)  is  useful  for  detecting  the  presence  of 
outliers,  modes  and  the  existence  of  two  populations.  Flat  slots  in 
Q(u)  indicate  modes.  Sharp  rises  in  Q(u)  for  u  near  0  or  1  suggest 
outliers;  sharp  rises  in  Q(u)  within  the  boxes  indicate  the  existence  of 
two  (or  more)  populations.  In  obvious  bimodality  shown  in  Figure  3  is 
represented  by  the  Quantile  Box  Plot  displayed  in  Figure  4. 
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Weibull  Distribution  Function 


The  M.L.  method  is  applied  in  order  to  obtain  the  two  parameters  of 
the  Weibull  function' 


f(x)  ■  ”  ($F  exp  [-  ($)  ]  (10) 

The  method  requires  defining  the  likelihood  function 


where  ■  data, 

m,  u  ■  shape  and  normalising  parameter  and 
N  ■  sample  site, 

By  solving  the  following  log  likelihood  equations 
-  0  and 

link  -  o  U2) 

om 

determines  the  &  and  0  values. 

i 

Equation  (12)  must  be  solved  in  an  iterative  manner  where  the 
initial  estimates  are  obtained  from  the  method  of  moments.  The  unbiased 
m  and  u  and  their  corresponding  confidence  intervals  are  obtained  from 
Tables  by  [8]. 

Wolbull  Size  and  Stress  Distribution  Relationship 

Tho  basic  equation  for  predicting  mean  failure  stress  o  of  the 
ring  from  mean  failure  stress  3,,  of  flexure  tests  is  p 
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^.-juaa  aiaiHiaa  KaUfflciditi  >• 


kdMMUiito 


where  a  A.  dependence  is  assumed  to  regulate  change  in  the  failure  stress, 
m^  is  thentfeibull  shape  parameter  obtained  from  the  flexure  data. 

The  and  &.  are  the  volumes  or  areas  of  the  ring  and  flexural  bar 
respectively*  depending  on  whether  surface  or  volume  flaw  theory  is  desired. 
Determination  of  K,  and  K.  depends  on  the  results  from  integrating  the  risk  of 
rupture1*  relation  4 


dii  «« 

such  that 

R  -  Mj  (frf 

Note, u*  ■  u(Ki,)  i,/m  ,  whore  u  is  defined  in  equation  (12)  end  that  K.-l 
for  ring,  since  it  Is  assumed  to  be  a  negligible  stress  gradient  through  4 
ring  thickness. 


Statistical  Flaw  Distribution  Theory  (Alternative  Method! 


An  alternative  surface  dependent  relation  was  applied  to  the  flexure 
data  in  order  to  predict  mean  failure  stress  of  the  ring.  This  method  in  a 
prior  application*  successfully  predicted  tensile  failure  of  96  percent 
alumina  cylindrical  rods  from  similar  material  and  geometry  using  four 
point  flexure  test.  The  maximum  stress  region  was  considered  in  this 
application.  The  size  dependent  relation  for  menu  failure  stress  o 
prediction  of  ring  is  P 


(15) 


where  is  determined  from  solution  of 


1 10) 


3H7 


The  ratio  —  is  the  coefficient  of  variation  (C.V.)  from  flexure  tests 
°1 

and  o.  and  S,  are  mean  failure  strength  and  standard  deviation  of  flexure 
test  fesultslrespectively. 

A.  and  A,  are  areas  of  components  subjected  to  maximum  stress.  This 
is  a  reasonable  assumption  for  the  requirements  in  the.ring  prediction 
process  since  elementary  fracture  mechanics  evaluation  indicates  that  a 
flaw  on  outer  ring  surface  would  have  to  be  23  percent  greater  than  on  the 
inner  ring  surface  for  an  equal  chance  of  failure.  Note  with  sizable 
increases  in  K,  the  difference  in  surface  areas  of  ring  and  flexure  bar 
will  result  in  minimum  effect  for  predicting  failure. 

RESULTS  AND  DISCUSSIONS 

In  Figure  5,  Pf  (probability  of  failure)  vs.  strength  results  for 
billet  number  2001  are  shown  for  both  the  ring  (I)  and  flexure  test  (II) 
results.  Application  of  the  Quantile  Box  Plot  shown  in  Figure  6  indicates 
bimodal  distribution  for  the  flexure  test  results.  The  mode  representing 
larger  strength  values  were  removed.  This  is  justified  since  ceramic 
material  strength  data  is  usually  represented  by  an  extreme  value  distribution 
(e.g.  Weibull)  which  is  skewed  to  the  left.  An  additional  reason  for 
excluding  the  second  mode  resulted  from  the  materials/test  laboratory 
responsible  for  the  data  indicated  that  difficulties  existed  with  regard  to 
the  test  fixture  used  in  obtaining  the  flexure  results.  Note  the  ring 
results  (I)  were  generally  represented  by  a  relatively  smooth  curve  with 
either  no  outliers  or  very  few  as  shown  in  Figure  5.  The  type  of  test  and 
the  component  geometry  probably  contributed  to  this  situation.  Note,  the 
Weibull  shape  parameters  (Modulus  M)  of  5.85  and  12.2  respectively  for  the 
flexure  and  ring  data  respectively,  differed  considerably  when  comparing 
the  two  test  results.  These  results  should  correlate  reasonably  well 
otherwise  application  of  the  size  effect  relations  described  previously  is 
not  valid.  That  is,  if  materials  are  similar  their  dispersion  values 
resulting  from  strength  tests  should  be  similar. 

In  Figure  7,  the  results  from  removal  of  second  failure  mode  are  shown 
as  II,  This  uni -modal  representation  of  flexure  data  provides  a  relatively 
smooth  curve  consistent  in  slope  and  appearance  with  the  probability  ranked 
ring  stress  failures.  The  agreement  between  the  dispersion  constants  (m) 
is  quite  acceptable,  therefore,  allowing  application  of  statistical  flaw 
distribution  theory,  where  it  is  assumed  that  material  and  uniformity  of 
failure  locations  are  similar  for  the  two  specimen  geometries. 

In  Figure  8,  the  P.  results  for  billet  No.  4001  is  shown.  In  this 
case,  outliers  contaminated  the  data  as  shown  in  Quantile  Box  Plot  (see 
Figure  9) .  Although  the  Huber  and  Andrews  robust  methods  for  determining 
outliers  is  usually  acceptable,  the  authors  considered  the  Quantile  Box 
Plot  more  desirable.  The  arbitrariness  in  selecting  tuning  constants  and 
scale  parameters  when  applying  the  Huber  Technique  are  the  primary  reasons 
for  relying  on  this  simpler  and  more  efficient  Quantile  Box  Plot  procedure, 
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Since  the  Quantile  function  must  initially  be  considered  in  determining  multi¬ 
modality,  then  it  is  a  simple  matter  to  complete  the  investigation  using  the 
plot  results.  Additional  results  (Billets  1001  and  3001)  are  shown  in  Figure 
10  and  11. 

The  results  of  predicting  failure  of  ring  from  flexure  test  results  are 
shown  in  Tables  1,  2  and  3.  In  Table  1,  a  comparison  of  m  values  indicates 
general  agreement  between  flexure  and  ring  results  when  the  robust  scheme  is 
applied.  In  the  conventional  Weibull  prediction  process  the  predicted  mean 
failure  stress  of  ring  using  robust  data  (T^)  agreed  somewhat  better  with 

T 

the  actual  ring  failure  results  (Sr,  )  than  the  corresponding  nonrobust  case 
(o„) .  '  If  the  maximum  stressed  region  is  considered  then  the  absolute  percent 
difference  E^,  is  reduced  considerably  when  applying  the  robust  method  as 

compared  to  nonrobust  considerations  (E^) .  ERTA  (total  areas  with  robust 

data)  Indicates  poor  correlation  for  all  billets  where  minimum  of  20  percent 
is  noted  for  2001  series.  From  these  results,  it  appears  obvious  that  total 
area  consideration  is  a  poor  choice  in  size  effect  failure  prediction  even 
with  robustness.  The  results  shown  in  Table  2  indicate  Weibull  volume  depend¬ 
ency  relationships  are  not  applicable  in  the  predictive  process  since  a 
minimum  percent  difference  of  62  exists  when  considering  any  of  the  four 
billets. 

The  results  from  applying  the  alternative  method2  with  robust  data  are 
shown  in  Table  3.  In  general  the  method  provided  slightly  better  predictive 
qualities  (E  )  than  the  Weibull  approach  when  considering  predicted  and 
actual  mean  ring  failure  stress.  The  C.V.  values  did  not  correlate  well  for 
the  3001  series,  predictive  results  were  fair  (11%  error)  indicating  that 
actual  equality  of  C.V.  or  m  are  necessary  in  applying  these  predictive 
methods  if  robust  data  is  used. 

CONCLUSIONS 


This  paper  has  described  a  predictive  process  that  successfully  determined 
mean  failure  of  a  large  S.C.F.S.  component  (Hydroburst  Ring)  from  flexure 
tests  on  small  specimens.  The  statistically  size  dependency  relations  of  the 
Alternative  Method  provided  acceptable  results  with  the  Weibull  method  less 
uesirable.  The  primary  requirements  are:  the  removal  of  outliers  (bad  data) 
or  bi-modal  distributions  by  formal  statistical  procedures  and  considering 
only  maximum  stressed  surfaces  in  the  predictive  process  for  this  particular 
material  and  manufacturing  process.  Results  have  indicated  that  S.C.F.S.  is 
not  a  volume  or  total  area  sensitive  material  in  predicting  its  failure.  It 
has  also  been  demonstrated  that  application  of  the  robustness  methods  deter¬ 
mines  more  realistic  material  characteristics  in  regards  to  the  failure 
stresses.  Bad  data  (outliers)  or  multi-modality  tend  to  distort  the  statistical 
model  in  providing  satisfactory  failure  prediction  methods.  The  Quantile  Box 
Plot  provided  a  more  desirable  method  for  obtaining  outlier*  ,*nd  multi-modality 
in  the  data,  The  Huber  and  Andrews  procedures  were  less  desirable  because  the 
arbitrariness  in  selecting  the  tuning  constants  and  scale  measure.  Symmetry 
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assumptions  prevented  recognition  of  a  separate  set  of  outliers  from  either 
of  the  tail  regions  of  the  distributed  data  in  addition  to  not  recognizing 
multi-modality  in  the  data. 
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TAILS  I  -  COMPARISON  MEAN  FAILURE  STRESS 

PREDICTED  VS.  ACTUAL  (ALTERNATIVE  METHOD) 
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THE  TRASANA  TERRAIN  RESEARCH  PROGRAM 


Warren  K.  Olson 
D.  Hue  McCoy 

US  Army  TRADOC  Systems  Analysis  Activity,  WSMR,  NM 


1.  Introduction.  During  the  last  decade,  the  US  Army  has  grown  Increasingly 
aware  of  a  need  to  fight  outnumbered  and  win.  Its  field  manuals  and  training 
literature  are  filled  with  suggestions  regarding  methods  of  generating  "combat 
multipliers"  {Ref.  1).  One  combat  multiplier  often  discussed  Involves  taking 
maximum  advantage  of  battlefield  terrain  In  the  deployment  of  defensive  and 
offensive  forces.  The  extension  of  this  concept  Involves  the  denial  of 
advantageous  terrain  to  the  enemy.  The  TRASANA  Terrain  Research  Program 
embraces  this  concept  through  the  development  of  computer  algorithms,  terrain 
data  bases,  and  operations  research  methodology  which  can  be  applied  In  the 
analysis  of  tactical  alternatives  and  weapons  systems  effectiveness.  The 
program  also  seeks  to  Improve  the  the  generalized  understanding  of  the 
operational  limitations  Imposed  by  the  battlefield  environment. 

This  paper  discusses  various  computer  programs,  color  graphics  aids,  and 
digital  terrain  data  bases  which  are  used  1n,TRASANA's  day-to-day  analysis 
activities.  New  methods  of  developing  and  digitizing  scenarios  for  Insertion 
In  Battalion  combat  models  are  presented.  Case  studies  of  the  effects  of 
terrain  on  battlefield  activities  are  also  provided.  Finally,  current 
research  efforts  aimed  at  classifying  terrain  and  generalizing  Its  effects  on 
Intervlslblllty  and  mobility  are  discussed, 

2.  Terrain  Analysis  Programs.  Most  of  TRASANA' s  terrain  analysis  requirements 
center  around  the  need  to  understand  Intervlslblllty  conditions  connected  with 
the  Army's  current  and  future  target  acquisition  and  weapons  delivery  systems. 
These  analyses  are  made  more  challenging  by  Army  requirements  to  operate  in 
many  theaters  around  the  globe,  and  use  both  airborne  and  ground-mounted 
surveillance  and  weapons  systems  In  accomplishing  Its  missions.  The  computer 
algorithms  most  frequently  used  In  defining  llne-of-slght  (LOS)  conditions  and 
In  visualizing  battlefield  terrain  are  described  below.  Many  of  these 
programs  are  documented  In  Reference  2.  However,  other  specialized,  less 
frequently-used  algorithms  exist  for  digitizing  Information,  preparing  and 
collecting  data,  processing  information,  and  In  providing  rapid  analysis  for 
study  support. 

a.  Optimum  Vantage  Point  LOS  Algorithm  (0PT1) 

This  program  uses  Defense  Mapping  Agency  (DMA)  digitized  terrain  with 
vegetation  and  urban  development  codes  as  a  computational  data  base.  The 
program  generates  line  printer  contour  maps,  vegetatlon/urban  code  maps,  end 
llne-of-slght  (LOS)  maps  In  a  UTM  grid  reference  system.  The  user  may  specify 
the  sensor/target  location  and  altitude  for  three  different  sensor/target 


This  article  also  appear*  in  the  proceedings  of  the  21st  Army  Operations 
Resaarch  Symposium. 
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platforms  (ground,  hellcopter/nap-of-the-earth,  and  fixed  wing  aircraft/ 
constant  altitude).  The  LOS  data  Is  stored  for  use  In  a  more  sophisticated 
program  (0PT2)  which  examines  the  unions  and/or  Intersections  of  coverage  from 
one  or  more  sensors. 

b.  Optimum  Vantage  Point  Pass  2 (0PT2 ) 

0PT2  uses  the  output  from  0PT1  as  Input.  The  program  Is  capable  of 
producing  LOS  maps  for  any  or  all  of  the  sensors  preprocessed  by  0PT1.  0PT2 
can  examine  the  union  and  Intersections  of  any  set  of  sensors  with  any  second 
set  of  sensors.  This  Is  particularly  useful  for  examining  the  coverage  of  a 
set  of  defender  weapons  against  an  expected  area  threat.  The  program  can  be 
used  to  restrict  the  view  of  a  given  sensor  to  ar\y  direction  with  a  limited 
field  of  view.  All  outputs  from  this  program  can  be  directed  to  the  CALCOMP 
plotter  or  Color  Graphics  System  and  scaled  to  fit  1:50,000  maps  (see 
Figure  1). 

c.  Point  to  Point  LOS  (PT2PT) 

Point  to  Point  calculates  LOS  from  a  discrete  point  to  a  set  of  target 
points.  This  program  is  used  In  the  calculation  of  advance  route/fllght  path 
exposure  lengths  and  associated  time  durations  from  any  given  senior.  The 
output  can  be  directed  to,  CALCOMP  plotter  to  give  a  profile  of  the  terrain 
between  the  sensor  and  the  target.  This  profile  will  graphically  show  the 
presence  of  vegetation  and  urban  features  (see  Figure  2), 

d.  SEEFAR  Model  (AMSAA) 

SEEFAR  Is  an  Improved  model  for  producing  11  ne-of-slght  maps.  Many 
models  determine  whether  a  target  Is  within  view  by  mathematically 
constructing  a  terrain  profile  between  observer  and  target  and  examining  It  to 
see  If  It  Interferes  with  1 1 ne-of-slght ;  this  requires  generating  a  completely 
new  profile  for  each  target  position.  SEEFAR  avoids  this  time-consuming 
profile  generation  by  dynamically  recording  the  characteristics  of  a  "running 
horizon"  as  computations  are  made  for  points  further  away  from  the  observer. 
For  each  target  point,  a  check  Is  made  to  determine  whether  the  target  Is 
behind  the  "horizon".  This  new  approach  results  In  a  dramatic  savings  In  both 
storage  and  computing  time  requirements.  SEEFAR  Is  Ideally  suited  for 
air/ground,  large  area  analyses  (see  Figure  3), 

e.  Plot  Contour. 

This  program  plots  topographic  contour  maps  from  digital  data,  and 
Includes  vegetation  and  urban  cover.  The  program  Is  useful  In  both  quality 
assurance  checks  of  the  loaded  data  base,  and  In  the  simulation  of  battlefield 
activities  (see  Figure  4). 
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Figure  1  0PT2  Program  output 
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Figure  2  PoInt-to-PoInt  Output 
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Figure  4  PLOT  CONTOUR  Topographic  Map 


f.  TRPGRFX, 

This  Color  Graphics  program  displays  elevation  Intervals  In  colored 
bands;  displays  vegetation  and  urban  features  In  appropriate  colors;  draws 
observer  (defender)  locations  and  attacker  routes;  generates  LOS  to  an  area  or 
along  routes  only;  provides  a  zoom-in  capability  for  high  resolution  work; 
generates  LOS  statistics  (to  Include:  probability  of  LOS,  In-view/ 
out-of-view  distributions,  first  sighting  range,  expected  opening  range,  and 
average  In-vlew/out-of-vlew  segments  per  path);  generates  multiple  attacker 
routes  (using  formations),  and  develops  attacker  position  vs.  time,  with  an 
LOS  Indicator  for  post  processing  other  statistics  (see  Figures  5-9). 

g.  CALCOMP  3-D  Package. 

The  program  draws  3-0  grldded  representations  of  the  earth's  surface 
with  or  without  perspective,  viewed  i'rom  the  surface  Itself  or  from  a 
defined  altitude  (see  Figure  10). 

h.  3-D  Terrain  Gray  Scaling  Progam, 

This  program  produces  a  3-D  perspective  Image  of  digital  terrain  data 
by  defining  both  a  sun  angle  and  viewing  angle,  The  view  on  the  Color  Graphics 
resembles  an  oblique  aerial  photograph  of  an  actual  terrain  surface  (see 
Figure  11). 
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hPGKFX  Elevation,  Vegetation,  Urban  Display 


Figure  6  TRPGRFX  Observer  Position  Display 


Figure  8  TRPGRRC  Une-of -Sight  Display 
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Figure  9  TRPGRFX  Zoan-In  Display 


Figure  11 


i.  LOS  Graphics. 


This  program  uses  digitized  terrain  data  with  vegetation  and  urban 
development  heights  as  a  computational  data  base.  The  program  produces 
contour  maps,  vegetation  plots,  and  road  nets,  within  a  UTM  grid  reference, 
using  QPT1  as  a  driver  model.  LOS  maps  are  produced  almost  Instantaneously. 
The  effects  of  smoke  on  visual  LOS  can  be  readily  displayed.  The  program  also 
produces  composite  LOS  maps  which  show  the  union  and  Intersection  of  LOS  from 
different  sensor  locations  (see  Figure  12). 

j.  CARMONETTE  History  Display  Program. 

This  real  time,  Interactive  graphics  program  displays  the  terrain, 
position  of  fighting  units,  firing  events,  and  kills  from  a  CARMONETTE  history 
file.  It  can  also  be  used  to  generate  llne-of-slght  maps  using  the  observer 
locations  and  the  CARMONETTE  land-deck.  The  program  Is  used  to  develop 
scenarios  and  analyze  battle  outcomes  (see  Figures  13-16). 

k.  Data  Base  Display/DMA  Merge. 

This  Interactive  program  package  "splices"  multiple  DMA  terrain  data 
tapes,  by  allowing  an  analyst  to  see  the  entire  data  library  while  seated  at 
the  Color  Graphics  console.  The  analyst,  through  joy  stick  or  keyboard  entry, 
specifies  the  region  desired  for  analysis  purposes.  The  package  then 
automatically  prepares  the  data  file  for  use  with  other  programs. 


In  addition  to  these  programs,  the  Agency  maintains  a  limited  capability  to  do 
mobility  processing  with  the  Army  Mobility  Model  (AMM76).  However,  most  of 
the  required  mobility  Information  Is  processed  for  TRASANA  by  the  Waterways 
Experiment  Station,  Vicksburg,  Mississippi. 

3.  Computer  Color  Graphics.  A  RAMTEK  Color  Graphics  System  (CGS)  Installed 
In  1^77  Interfaces  with  the  TRASANA  UNIVAC  1100/82  Mainframe  computer.  The 
TRASANA  Computer  Graphics  Facility  (CGF)  Is  an  applications/user  oriented 
Ineractlve  color  graphics  system.  The  hardware  for  the  system  Includes  a 
RAMTEK  9300  color  raster  graphics  system  with  associated  devices  (trackballs, 
joysticks)  Including  four  display  units,  a  SAC  sonic  digitizer,  and  a  VARIAN 
V- 77  minicomputer.  The  software  Includes  a  CGF  applications  library,  plus  an 
Interface  that  permits  use  of  the  CALCQMP  graphics  library,  The  user  programs 
reside  on  the  1100/82  and  the  graphics  facility  programs  reside  on  the  V- 77  In 
an  effort  to  distribute  the  workload  for  the  CGF  and  therefore  enhance  the 
rate  of  response  for  user  applications.  This  system  Is  used  for  terrain 
analysis,  detailed  scenario  preparation  and  many  other  analysis  applications. 
The  system  components  are  shown  In  Figures  19  and  20. 

4.  Digital  Terrain  Library.  Most  of  the  ulgltlzed  terrain  data  used  for 
study  purposes  comes  from  two  sources.  The  topographic  Information  (to 
Include  elevation  data  at  12.5m  Intervals,  vegetation,  and  urban  features 
data)  are  provided  by  the  Defense  Mapping  Agency,  Washington,  O.C.  For  some 
applications,  Including  detailed  combat  modeling,  additional  Information 
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Figure  12  Line-of-Sight  Graphics  Display 
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1  UNIVAC  1108  HOST  COMPUTER 
1  VAftlAN  77-613  MINI  COMPUTER 
5  VARIAN  620/1  MINI  COMPUTERS 
4  MONOCHROMATIC  DISPLAYS 
4  RAMTEK  COLOR  DISPLAYS 
4  TRACK  BALL/JOY  STICK 
1  GRAPH  PEN-3  DATA  TABLET  (36  x  36") 
1  ELECTROSTATIC  PRINTLR/PLOTTER 


Figure  20  Color  Graphics  Facility 
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concerning  cover,  concealment,  and  on-and  off-road  trafficablllty  are 
required.  This  data  Is  purchased  under  contract  from  the  Waterways  Experiment 
Station.  Detailed  data  are  available  for  several  regions  around  the  world 
(Ref.  3).  Some  of  the  available  regions  are  shown  In  Figure  21.  Other  DoD 
agencies  and  contractors  are  currently  Involved  in  the  development  of  data  on 
the  structure  of  cities  for  use  In  studies  of  military  operations  in  urban 
terrain  (MOUT).  The  Defense  Mapping  Agency  will  shortly  develop  a  new 
prototype  land  combat  data  base  for  evaluation  (see  Appendix  A).  If  approved, 
the  new  data  base  would  provide  analysts  with  heretofore  unavailable  digital 
data,  addressing  surface  configuration,  surface  features,  surface  materials, 
hydrography,  movement  and  other  features  (approximately  200  bits  of 
Information  would  be  stored  par  grid  point).  A  DMA  prototype  land  combat  data 
base  for  Ft.  Lewis,  WA  will  be  available  for  testing  In  November  1381. 

5.  Test  Applications.  With  these  tools  and  data  bases,  It  Is  possible  to 
Investigate  many  aspects  of  the  1 ntervl s 1b1 1 1 ty  “combat  multiplier"  without 
programming  expensive  field  tests,  scheduling  manpower  and  equipment, 
experiencing  delays  due  to  weather,  and  facing  a  host  of  other  problems. 
Rather,  In  the  comfort  of  a  computing  laboratory,  the  analyst  can  easily 
duplicate  many  of  the  measurements  made  In  previous  field  tests  such  as  the 
NATO  Range  Study,  Lost  Horizons,  TETAM,  HELAST,  and  CHINESE  EYE.  These 
measurements  and  studies  can  be  completed  in  days  or  weeks,  as  opposed  to  the 
prevous  yard  stick  of  months  or  years.  Also,  It  Is  possible  to  analyze 
regions  of  the  earth's  surface  which  are  not  readily  accessible  for  field 
testing,  either  due  to  distance,  land  use,  or  political  considerations. 


In  order  to  feel  comfortable  with  the  results  of  these  simulations,  however  It 
must  be  demonstrated  that  the  computer  algorithms  and  data  bases  are  capable 
of  closely  matching  field  test  results.  In  the  late  I960' s  and  early  1970 * s 
some  effort  was  spent  comparing  computer  predictions  of  1 ntervl s Ibll 1  ty  with 
major  field  test  results.  The  findings  were  somewhat  mixed,  but  held  promise. 
Difficulties  with  the  documentation  of  the  field  tests  and  digitized  terrain 
resolution  were  noted,  and  modifications  to  some  of  the  computer  algorithms 
appeared  warranted. 

Recently,  TRASANA  had  the  opportunity  to  access  some  high  resolution  field 
test  results  developed  by  the  Combat  Development  Experimentation  Command 
(CDEC)  during  a  telemetry  test  at  the  Army's  new  training  facility  at  Ft. 
Irwin,  California  (Ref.  4).  In  the  referenced  study,  while  using  the  terrain 
analysis  programs  to  optimize  an  RF  posltlun/locatlon  system,  TRASANA  was  also 
able  to  validate  these  models  with  actual  telemetry  data  (See  Figure  22). 

This  study  Is  reported  In  detail  In  a  separate  1981  AORS  paper.  It  Is 
sufficient  here  to  note  that  88  percent  agreement  between  measured  and 
simulated  results  was  demonstrated.  This  is  a  remarkable  achievement, 
considering  the  nature  of  the  terrain  and  the  fact  that  the  telemetry  system 
which  produced  the  test  data  has  a  slight  (but  unquantified)  capability  to 
"see"  beyond  the  horizon  and  through  veqetatlon.  In  the  Ft.  Irwin  study  and  a 
companion  study  just  completed  for  the  u  Sill  HEL3AT  8  Test  (Ref.  5),  the 
LOS  programs  have  also  been  used  to  grade  coverage  quality,  to  “orthogonal Ize" 
positions  In  order  to  minimize  target  location  error,  and  to  optimize  coverage 
with  a  minimum  Investment  In  equipment  (see  Figure  23). 
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Figure  21  Digitized  Terrain  Display  -  West  Germany 
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Figure  23  Fort  Sill  Telemetry  Grading  Analysis  (11  Sites) 
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6.  Terrain  Analysis  Studies.  The  TRASANA  Terrain  Research  Program  Is  also 
used  to  support  a  wide  variety  of  Army  studies,  ranging  from  weapon  systems 
design,  to  studies  of  tactics,  reliability,  and  trade-off  analyses  between 
competing  systems.  Preprocessed  Information  Is  also  developed  for  use  in 
higher-level  models.  Several  examples  are  provided  below. 

a.  ATGM  Design. 

One  class  of  problem  which  occurs  periodically  concerns  the 
appropriate  range  capability  for  a  given  type  of  antitank  guided  missile 
(ATGM).  In  a  recent  analysis,  16  scenarios  were  evaluated  with  regard  to  LOS 
and  firing  opportunity  (Figure  24).  Distributions  of  engagement  ranges  and 
Intervlslblllty  lengths  ware  developed  for  European  and  Middle  Eastern 
scenarios.  The  results  show  that  Increased  engagement  opportunities  could  be 
achieved  by  extending  the  range  of  the  medium  range  ATGM  from  1000  to  2500 
meters;  however,  the  value  of  Increasing  the  range  of  long-range  (heavy)  ATGMs 
beyond  4000m,  especially  In  the  presence  of  smoke  Is  doubtful  (Ref.  6). 

b.  Main  Battle  Tank  Reliability. 

Addressal  of  the  significance  of  Ml  tank  reliability  test  data 
recently  required  the  analysis  of  distance  moved  on  the  battlefield  during 
combat  (see  Figure  25).  Analysis  of  fourteen  European  scenarios  shows  that 
the  distribution  of  movement  distances  Is  log  normal  with  a  mean  of  10.0km 
per  move  (Ref.  7). 


c.  IFV/CFV  Fighting  Tactics. 

5tud1es  have  been  done  which  address  the  advantages  of  using  pop-up 
tactics  to  limit  exposure  of  the  new  class  of  fighting  vehicles.  It  has  been 
found  that  considerable  natural  defilade  exists  for  use  as  fighting  positions, 
that  the  pop-up  tactic  enhances  survivability  without  a  major  degradation  In 
target-servicing  capability,  and  that  based  on  the  typical  site  of  the  killing 
zones,  a  four-missile  launcher  is  desirable  to  minimize  crew  exposure  and  time 
away  from  position  during  reload  (Ref.  8). 

d.  SOTAS/PAVE  MOVER  Intervlslblllty  Modeling. 

Questions  concerning  the  level  of  detail  required  In  air-to-ground 
Intervlslblllty  analyses  In  northern  Europe  and  the  need  to  provide 
intervlslblllty  Inputs  to  Division-  and  Corps-level  wargames  prompted  a 
research  effort  which  In  part  analyzed  the  level  of  correlated  Intervlslblllty 
from  laterally  separated  aerial  target  acquisition  positions  (see  Figure  26). 
It  was  concluded  that  In  the  NQRTHAG  area,  medium-altitude  aerial  surveillance 
can  be  modeled  reasonably  well  by  selecting  one  general  platform  position 
which  represents  the  position  of  the  device  within  a  Division  area.  Multiple 
positions  separated  from  each  other  laterally  by  as  much  as  20km  still  exhibit 
LOS  probabilities  with  correlation  coefficients  of  0,72  -  0.83,  with  a  mean  of 
0.78,  Reasonable  surveillance  Is  possible  In  this  area  using  a  mission 
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altitude  of  approximately  1500m,  AGL.  It  was  found  that  target  exposure  Is 
sensitive  to  the  type  of  vehicle  movement  —  cross-country  or  on-road. 
High-altitude  surveillance  missions  of  the  PAVE  MOVER  variety  can  be  modeled 
reasonably  well  using  the  SOTAS  results  for  look-down  angles  between  1.0  and 
9.0  degrees.  This  study  Is  now  being  extended  Into  the  CENTAG  region,  since 
It  is  expected  that  the  rougher  terrain  In  the  south  may  change  statistics  and 
operating  altitudes  considerably  (Ref.  9). 

7.  Terrain  Classification  Research.  The  above  discussion  captures  the  varied 
flavor  of  the  terrain’ analyses  ctfto’n  required  to  support  US  Army  studies,  and 
points  to  a  need  to  generalize  results  whenever  possible.  Because  of  this 
requirement,  an  International  working  party  was  formed  In  1978  to  share 
Information  and  then  to  sue*  ways  of  classifying  terrain  and  Its  .effects  on 
military  operations.  The  present  effort  emphasizes  the  causal  relationships 
which  drive  the  results  of  Intervlslblllty  and  mobility  analyses.  In  order  to 
assist  the  development  of  a  workable  Intervlslblllty  classification  system, 
TRASANA  has  embarked  on  a  30  man-month  research  program  designed  to 
Investigate  the  relationships  among  terrain  geometry.  LOS  statistics  and 
predicted  battle  outcome.  The  TRASANA  study  (see  Figure  27)  will  utilize 
digital  topography,  Intervlslblllty  computer  programs,  and  the  CARMONETTE 
battalion  combat  simulation  In  an  attempt  to  establish  Intervlslblllty 
relationships  within  the  context  of  military  scenarios  for  selected  regions  of 
West  Germany.  This  approach  was  developed  based  on  experience  gained  In  prior 
US  studies  (see  Refs.  10-14).  The  goal  of  the  study  Is  tp  Identify  a 
classification  scheme  based  on  topography  and  surface  clutter  which  can  be 
used  to  predict  Intervlslblllty  conditions  for  a  wide  range  of  military 
systems. 


LENGTH  OF  TANK-TRAVERSED  SEGMENTS  X  FREQUENCY  OF  LENGTH  OCCURANCES 
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Figure  28  Distribution  of  Tank-Traversed  Path  Lengths 
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Figure  26  SOTAS/PAVE  MOVER  LOS  Correlation  Analysis 
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a.  Test  Areas. 

Seventeen  test  map  sheets  have  been  selected  based  on  the 
availability  of  appropriate  digital  data  and  the  variability  in  surface 
roughness  (See  Figures  28-30).  These  regions  can  be  grouped  Into  ten  test 
areas  (Figure  31)  using  the  1974  Natick  Landform  Classification  System,  which 
together  represent  nearly  55  percent  of  the  variability  In  surface  roughness 
for  West  Germany.  The  test  areas  allow  limited  comparisons  between  two  or 
more  map  sheets  representing  the  same  terrain  "type1*.  They  also  permit  the 
comparison  of  statistics  generated  via  computer  models  with  those  developed  In 
certain  field  trials.  Visualizations  of  the  variability  in  surface  roughness 
are  shown  schematically  In  Figure  32  and  In  a  3-D  plot  of  the  Neumarkt  region 
In  Figure  33.  The  codes  for  the  Natick  Landform  Classification  System  (Ref. 
11)  are  provided  In  Figure  34. 

b.  Intervisibility  Classification  System. 

The  Natick  classification  system  depicts  the  typical  surface  roughness 
of  a  landform  compartment  (see  Figure  30)  by  describing  Its  maximum  local 
relief,  modal  local  relief,  and  number  of  positive  features  per  mile  along  a 
random  transect  through  the  compartment.  Together,  the  latter  descriptors 
present  a  sinusoidal  picture  of  the  typical  terrain  profile  In  a  compartment, 
modified  periodically  by  larger  hill  masses  ("outliers")  as  defined  by  the 
maximum  local  relief.  Fof'  areas  free  of  ary  surface  clutter,  there  appears 
to  be  some  positive  correlation  between  landform  type  and  Intervisibility 
statistics.  However,  most  Intervisibility  Is  further  modified  by  vegetation 
and  cultural  features  (urban  areas)  which  appear  on  maps,  but  which  are  not 
described  by  the  Natick  system. 

To  remedy  this  difficulty,  a  tentative  vegetatlon/urban  classification  system 
has  been  developed  (see  Figure  35)  which  Is  patterned  after  the  structure  of 
the  Natick  System.  The  TRASANA  system  for  classifying  this  surface  clutter 
uses  three  additional  Identifiers  to  describe  vegetation  and  urban  features  in 
terms  of  their  median  height,  median  thickness,  and  median  separation.  The 
Information  required  to  classify  a  region  In  this  manner  is  developed  through 
computer  processing  of  DMA  digital  topographic  data.  The  computer  programs 
allow  the  classification  of  vegetation  and  urban  features  either  separately  or 
jointly  and  provide  statistics  useful  In  the  analysis  process  (see  Figures 
36-39).  The  map  Information  summarized  In  Figure  40,  plus  10S  statistics  will 
be  used  to  test  and  modify  the  above  classification  system  as  necessary  during 
the  course  of  the  study.  A  preliminary  statistical  design  is  provided  In 
Appendix  B. 

c.  Tactical  Scenarios, 

Since  a  major  goal  of  the  classification  study  Is  to  develop  a  system 
which  can  predict  military  1 ntervl si bl 1 1 ty  conditions  without  large  scale 
field  trials  or  massive  computer  simulations,  It  Is  Important  that  the  LOS 
statistics  to  be  used  In  the  development  of  the  classification  system  reflect 
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Figure  29  Terrain  Classification  Map  Sheets 
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NEUMARKT 

(L6734) 


Figure  33  Neumarkt  Region,  showing  Landform  Compartments 


MAXIMUM  HILL  HEIGHT  (LOCAL  RELIEF) 
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6 
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MODEL 

HILL  HEIGHT  (LOCAL 

RELIEF) 

DESCRIPTOR 

CLASS  INTERVAL 

A 

0  -  10  METERS 

0-33  FEET 

D 

10  -20 

33  -  66 

C 

20  -35 

66  -  115 

D 
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115  -  165 
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75  -  100 

248  -  330 
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330  -  413 
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49 5  -  578 

J 

175  -200 

578  -  660 
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OVER  660 

NUMBER  Of  POSITIVE  FEATURES  P®  MILE 


DESCRIPTOR 


CLASS  INTERVAL 

O-QB  KILOMETERS 
0.8  -  1.6 
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3.2  -  4j0 
OVER  40 


0-0.5  MILES 
0.5  -  1.0 
1.0  -  15 
1.5  -  20 
20  -  2.5 
OVER  25 


Figure  34  Natick  tandform  Classification  Descriptors 


VEGETATION/URBAN  CLASSIFICATION  DESCRIPTORS 
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MEDIAN  HEIGHT 

CLASS  INTERVAL 
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Figure  35  Class  Intervals  -  Vegetation/Urban  Descriptors 
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MAP  STATISTICS 


1.  TOPOGRAPHY 

•  HIGH  POINT  (ELEVATION,  COORDINATES) 

•  LOW  POINT  (ELEVATION,  COORDINATES) 

•  MAP  RELIEF  (HIGH  -  LOW] 

2.  LANOFORMS 

•  MAXIMUM  HILL  HEIGHT 

•  MODAL  HILL  HEIGHT 

•  POSITIVE  FEATURES 

3.  VEGETATION 

•  PERCENT  VEGETATION 

•  NUMBER  OF  PATCHES 

•  MINIMUM  AREA,  DIMENSIONS 

•  MAXIMUM  AREA.  DIMENSIONS 

•  DIMENSION  DISTRIBUTION 

•  HEIGHT  DISTRIBUTION 

4.  URBAN  AREAS 

•  PERCENT  URBAN 

•  NUMBER  OF  PATCHES 

•  MINIMUM  AREA,  DIMENSIONS 

•  MAXIMUM  AREA,  DIMENSIONS 

•  DIMENSION  DISTRIBUTION 

•  SPACING  DISTRIBUTION 

•  HEIGHT  DISTRIBUTION 


Figure  40  Map  Statistics 
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a  valid  military  setting.  To  Insure  this,  detailed  tactical  scenarios  and 
overlays  are  being  prepared  for  each  test  area.  An  example  Is  provided  In 
Figures  41-45  for  the  Peine  Map  Sheet  (L3726).  This  scenario  portrays  a  BLUE 
covering  force  area  (CFA)  and  main  battle  area  (MBA)  battle  (Figures  42  and 
43).  Only  the  last  positions  of  the  units  In  the  CFA  are  Included.  The 
scenario  consists  of  one  cavalry  troop  In  the  CFA  and  one  tank  battalion  task 
force  In  the  MBA.  In  the  MBA,  the  battalion  is  deployed  with  two  company 
teams  forward  with  one  company  positioned  on  the  commanding  terrain  In  the 
center.  A  total  of  6  tank  platoons,  3  mech  platoons  and  3  cavalry  platoons 
are  available  to  the  force.  After  completing  the  covering  force  mission,  the 
cavalry  troop  Is  attached  to  the  tank  battalion  and  Is  attached  by  platoon  to 
the  companies  In  the  MBA. 

The  RED  attack  (Figures  44  and  45)  consists  of  a  reinforced  motorized  rifle 
regiment  attacking  on  two  axes.  The  attack  uses  three  battalions  abreast  with 
the  fourth  battalion  being  committed  In  the  center  after  seizure  of  the 
Initial  objectives.  The  RED  plan  depicts  the  route  of  advance  of  each 
battalion  and  the  formation  of  these  units  as  thqy  proceed  In  the  attack. 

The  overlay  originals  are  color-coded  and  keyed  with  the  respective  legends  to 
allow  the  considerations  of  all  likely  vehicle  attack  paths  and  occupied 
defensive  positions  during  the  Intervl slbl 1 1 ty  analysis. 

In  this  analysis,  It  is  desired  that  all  statistics  gathered  reflect 
Information  from  tactically  realistic  scenarios.  In  order  to  do  this,  the 
movement  of  an  attacking  threat  force  must  be  simulated.  That  Is  to  say,  once 
a  given  attacker  route  Is  specified,  the  movement  and  location  of  Individual 
vehicles  within  a  given  formation  Is  needed.  This  Is  referred  to  as  the 
"multiple  route"  problem.  Digitization  of  Individual  routes  was  tried  for  one 
scenario.  This  Is  not  a  feasible  solution  to  the  problem  for  several  reasons. 
First,  the  volume  of  data  Is  enormous.  Second,  movement  and  phasing  of 
Individual  vehicles  within  a  formation  Is  still  a  problem.  Therefore,  a 
computer  algorithm  was  generated  which  gives  the  time/position  history  for 
each  vehicle  along  a  given  attacker  route,  Basically,  the  routine  generates 
the  position  for  a  lead  vehicle  In  a  formation  and  maintains  the  Integrity  of 
that  formation  by  having  all  other  vehicles  follow  a  course  parallel  to  the 
attacker  route  at  constant  trailing  distance. 

d.  CARMONETTE  Analysis. 

A  subset  of  the  available  tactical  scenarios  will  be  prepared  for 
Insertion  In  the  CARMONETTE  battalion-level  Monte  Carlo  combat  simulation. 

The  Intent  of  this  effort  Is  to  develop  time  phased  engagement  results  which 
can  be  compared  with  LOS  statistics  to  examine  the  effects  of  Intervl slbl 1 1 ty 
on  battle  outcome.  Although  this  Is  an  Interesting  research  topic  In  Its  own 
right  (see  Ref.  12),  the  main  objective  of  this  effort  will  be  to  gain  Insight 
concerning  multiple  1 1 nes-of-s Ight  In  a  realistic  m-on-n  target  engagement 
environment.  These  results  will  be  useful  In  the  further  examination  of  LOS 
"coherence"  phenomena  (occasions  In  which  LOS  correlations  exceed  0.7). 
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Figure  42  BLUE  Force  Legend  (Peine) 
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Figure  44  RED  Force  Legend  (Peine) 
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e.  Tactical  LOS  Measures. 


During  the  study  several  tactical  1 ntervl Sibil Ity  measures  will  be 
developed  for  use  In  the  classification  effort,  as  well  as  In  extending  the 
knowledge  of  battlefield  characteristics  (See  Figure  46).  Some  measures 
(Figure  47)  are  classical  In  nature  and  have  been  used  In  a  variety  of  past 
trials  and  studies.  Others,  especially  ALOS/DLOS  and  coherence,  are 
relatively  recent  in  usage  and  give  promise  of  being  highly  correlated  with 

F19ures  48'52)‘  Additional  statistics  suggested 
by  the  CHINESE  EYE  and  KINGS  RIDE  field  trials  or  this  analysis  will  also  be 
examined. 


f.  Classification  Work  Outline. 

As  of  1  October  1981,  the  1 ntervl slbl 1 1  ty  classification  work  outlined 
Is  approximately  50  percent  completed.  Figure  53  Indicates  the  major  tasks 
and  shows  partially  (/)  or  fully  completed  (X)  work  for  each  of  the  ten  test 
areas.  It  Is  anticipated  that  the  remaining  effort  will  be  completed  and  a 
"commended  Interim  i ntervl slbl 1 1  ty  classification  systan  will  be  available  In 
M1d*1982e 


g.  Mobility  Classification. 

A  parallel  International  effort  concerning  the  classification  of 
terrain  with  regard  to  tactical  mobility  Is  also  In  progress.  This  effort 
will  use  a  screening  process  to  arrive  at  a  group  of  map  sheets  for  detailed 
analysis  (perhaps  utilizing  some  of  the  same  scenarios  generated  for  the 
1 ntervl slbl Ity  work).  From  this  work,  a  library  of  digitized  mobility  data 
developed  which  represents  specific  map  sheets  In  great  detail,  and 
which  will  be  related  through  factor  overlays  to  broader  regions,  for 
Inferential  purpses,  The  results  of  the  mobility  classification  research  are 
expected  In  1983, 

h.  Summary. 

Although  the  effort  outlined  above  Involves  a  difficult  problem  whose 
solution  has  evaded  researchers  for  several  decades,  TRASANA  believes  that 
the  scope  of  the  current  work  program  Is  broad  enough  and  the  tools  are  now 
available  to  permit  a  major  advancement  of  the  understanding  of 
1 ntervl slbll Ity  In  a  tactical  environment.  If  the  work  remains  uninterrupted, 
a  breakthrough  In  the  ability  to  classify  terrain  and  use  that  classification 
system  for  predictive  purposes  (see  Figure  54)  Is  anticipated.  Such  a  system 
would  have  multiple  uses  within  the  military  operations  research  community. 
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TACTICAL  LOS  MEASURES 

•  PROBABILITY  VS  RANGE 

•  IN-VIEW  DISTRIBUTION 

•  OUT-OF-VIEW  DISTRIBUTION 

•  OPENING  RANGE  DISTRIBUTION 

•  ALOS/DLOS 

•  CORRELATION 

•  COHERENCE 


Figure  46  Tactical  Llne-of-SIght  Measures 


PERCENT 


Figure  47  Functional  Curve  Fit  - 
LOS  Probability 
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Figure  48  Functional  Curve  Fit  - 
LOS  Probability 


PREMISE:  ANY  CLASSIFICATION  OF  TERRAIN  BY  EXPOSURE  CHARACTERISTICS 
IS  USEFUL  ONLY  TO  THE  EXTENT  THAT  IT  PROVIDES  INFORMATION 
RESARDIN6  THE  POTENTIAL  OUTCOMES  CPDFWIN)  OF  COMBAT  ON 
ALL  TERRAINS  IN  A  CLASS. 

CONCLUSIONS! 

1.  COMBAT  RESULTS  NOT  CORRELATED  WITH  AVERAGE: 

A.  EXPOSURE  LENGTH 

B.  PERCENT  OF  PATH  EXPOSED 

C.  NUMBER  OF  SIMULTANEOUS  EXPOSURES 

D.  LAST  COVERED  RANGE  (90S) 

E.  OPENING  RANGE 

F.  PLOS 

2.  PDFHIN  IS  STRONGLY  CORRELATED  WITH: 

A.  ALOS(N)/DLOS(N) 

B.  ATTACKER  EXPOSURE  COORDINATION 

3.  WEAPON  SYSTEM  STUDIES  DON'T  NEED  MULTIPLE  SCENARIO  TEAMS 
WORKING  ON  SAME  TERRAIN;  DO  NEED  MULTIPLE  SCENARIOS/TERRAINS. 


Flgura  50  Farrall  Terrain/Tactics  Study  Conclusions 
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ALOS(Nq)  -  MEAN  NUMBER  OF  SECONDS  (IN  RANGE  BAND)  !N  WHICH  AN  ATTACKER 
WEAPON  SYSTEM  WOULD  HAVE  Ng  OR  MORE  EXPOSED  TARGETS  AND 
WOULD  BE  TACTICALLY  PERMITTED  TO  FIRE. 

DtOS(N0)  -  MEAN  NUMBER  OF  SECONDS  (IN  RANGE  BAND)  IN  WHICH  A  DEFENDER 
WEAPON  WOULD  HAVE  Ng  OR  MORE  EXPOSED  TARGETS  AND  WOULD  BE 
TACTICALLY  PERMITTED  TO  FIRE. 

ALPS  .  (1-H)  MPjT  +  F 

DLOS  "  (1-H)  (Ml-P^IT Vf 

WHERE 

H  -  DEGREE  TO  WHICH  ATTACKER  USES  COVERED  ROUTES 

PI  -  FRACTION  OF  OPENING  RANGE  BAND  IN  WHICH  A  RANDOM  POINT  IN 
THE  AREA  (1KM  DEEP  AND  AS  WIDE  AS  THE  DEFENDER  FRONT)  IS 
WITHIN  TACTICAL  RANGE  OF  AND  HAS  L1NE-OF-SIGHT  TO  AT  LEAST 
101  OF  THE  DEFENDERS 

M  -  FRACTION  OF  TIME  ATTACKER  IS  ABLE  TO  SPEND  FIRING  WHILE 
ADVANCING 

T  •  TIME  REQUIRED  FOR  ATTACKER  TO  ADVANCE  1KM 

F  •  AVERAGE  AMOUNT  OF  TIME  ATTACKER  SPENDS  STOPPED  IN  FIRING 
POSITION 

P2  ■  AVERAGE  FRACTION  OF  THE  OPENING  RANGE  BAND  AREA  VISIBLE  TO 
RANDOM  DEFENDER 

C  *  ATTACKER'S  DEGREE  OF  EXPOSURE  COORDINATION/  A  NUMBER  BETWEEN 
1  AND  THE  NUMBER  OF  ATTACKERS  REPRESENTING  THE  AVERAGE 
NUMBER  OF  INDEPENDENT  TARGETS  PROVIDED  BY  THE  ATTACKER 


Figure  51  ALOS/DLOS  Definition 
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Figure  52  LOS  Correlation  Calculation 
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Figure  53  Terrain  Classification  Work 
Outline  (1  Oct  81) 


1.  EXTEND  LOS  INFORMATION  DERIVED  IN  PREVIOUS 
STUOIEB/TISTS  TO  NED  REGIONS  OP  SIMILAR 
SURFACE  GEOMETRY. 

S.  QUIOE  NED  DEAPONS  ACQUISITION  AND  FORCE 
TAILORING.  I 

S.  INCREASE  SURVIVABILITY  ON  THE  BATTLEFIELD. 

4.  PREDICT  LOS  CHARACTERISTICS  AS  FN  OF 
CLUTTER  AND  SURFACE  ROUOHNEISt 

f 

'Cn  (A  Coi«R  +  B  Sin  OR) 


Figure  54  lntervlslblllty  Classification  Uses 
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8.  Conclusions.  Many  advances  In  terrain  analysis  software  have  been  made 
since  the  late  1970s.  Visualization  of  the  battlefield  on  computerized  color 
graphics  Is  now  commonplace.  It  is  now  also  possible  to  develop  detailed  LOS 
masking  Information  complete  with  statistical  analysis  In  under  six  minutes  of 
computer  processing  time  for  regions  up  to  40  x  70km  In  size  (Refs.  9  and  15). 
This  represents  a  ten-fold  quantum  jump  In  processing  capability  which  did  not 
exist  two  years  ago.  The  accuracy  of  computer  predictions  has  also  been 
quantified  and  found  reasonable  for  most  applications.  As  processing 
capability  has  Improved,  so  has  data  base  coverage,  although  to  a  lesser 
degree.  Possible  near-term  development  of  a  new  Defense  Mapping  Agency  land 
combat  data  base  would  do  much  to  Improve  analysis  capabilities.  Longer  term, 
DMA  production  would  Improve  the  capability  to  do  land  combat  analysis  In  a 
wide  variety  of  locations  around  the  world.  Once  this  feasibility  Is 
demonstrated,  transfer  of  this  Intel  11  gence/operatlonal  planning  capability 
from  the  laboratory  to  tactical  units  In  the  field  Is  sure  to  follow. 

Mnally,  If  the  terrain  classification  research  program  Is  successful.  It  will 
be  possible  to  generalize  and  extend  analytic  efforts  to  other  regions  when 
time  or  lack  of  data  prevent  detailed  analysis  --and  do  It  with  confidence. 

The  decade  of  the  1980s  promises  more  Improvements  In  store  for  this  Important 
"combat  multiplier". 
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Best  Available  Com' 


APPENDIX  A 


26  August  1981 


PROTOTYPE  PORT  LEWIS-YA2CIMA  TRAILING  AREA* 
TACTICAL  TERRAIN  ANALYSIS  DATA  BASE  (TTADB)  FORMAT 
FOR  1:50,000  SCALE  PRODUCTS 


Data  Element 


Sit 

Designation 


I.  Surface  Configuration  Overlay 
Elevation  (m)  1  -  16 


Slope  (S) 


17  -  20 


#  of 
Rita 


(16) 

(4) 


Vegetation  Overlay 
Type 


21-26 


(6) 


Cade 


0 

1 

2 

3 

4 

5 

6 

7 


8-14 

15 


0 

1 

2 

3 

4 

5 


7 

8 
9 

10 

11 

12 

13 

14 

15 

16 


Value  Reoresentad 


No  Data 
0 
3 

10 
20 
30 


_  i 

-  10 
-  20 

-  30 

-  45 
>45 


Naturally  and/or  culturally 
dissected  land  (G->45) 
(Numerous  small  hillocks , 
sand  dunea,  glacial  debris, 
landfills,  dumps,  etc.) 

Open  Water 


No  Data 

Agriculture  (dry  crops) 

Agriculture  (wetland  rice) 
Agriculture  (terraced  crops, 
both  wet  and  dry) 

Agriculture  (shifting  cultivation) 
Brus hi and/Scrub  0C5o  high, 
nearly  open  to  medium  spacing) 
Brushiand/Scrub  «5o  high, 
medium  to  dense  spacing) 
Coniferous/Evergreen  Forest 
Deciduous  Forest 
Mimed  Forest 

Orchard/Plantation  (rubber, 
palm,  fruit,  etc.) 

Grassland,  Meadows,  Pasture 
Grassland  with  Scattered  Trees, 
Some  Scrub  Growth 
Forest  Clearings  (cutover  areas, 
burns,  etc.) 

Swamp  (mangrove ,  cypress,  etc.) 
Marsh/Bog  (peat,  muskeg,  etc.) 
Wetlands  (L.S.T.,  low-lying  wet 
areas ) 


•NOTE:  On  account  of  programming  tine  limitations,  this  Fort  Lewis-Takima  orototype 
is  a  condensation  of  the  full  proposed  Tactical  Terrain  Analysis  Data  Base.'  Numerous 
data  fields  have  had  to  be  comoressed,  omitted,  or  specifically  tailored  to  the  Fort 
Lawis-Yakima  terrain  conditions  to  meet  these  limitations. 


'.59  Best  Available  O 


Bit  #  of 

Data  Slesmt  »i<rrat',on  Bits  "■  '  *’■’  -?  "rnresented 

Type  (Ccn't)  -T  ■> .  rd/Hop-garden 

&  jaboo 

19  Dare  Ground 

20-22 

23  Open  Water 

24  Built-up  Areas 
25-29 

30-63  Mot  Used 

Canopy  Closure  (S)  27-29  (3)  0  Mo  Data 

1  0-25 

2  25-50 

3  50  -  75 

4  75-100 
5-7 

Tree  Height  (a)  30-33  (4)  0  Mo  Data 

1  0-2 

2  2-5 

3  5-10 

4  10-15 

5  15-20 

6  20  -  25 

7  25-30 

8  30  -  35 

9  >35 

10-15  Mot  Used 

Stan  Diameter  (a)  34-37  (4)  0  No  Data 


1 

0 

(Note:  CCM  formula  uses  meters 

2 

.00 

-  .02 

and  these  ranges  were  selected 

3 

.02 

-  .04 

because  they  best  correspond  to 

4 

.04 

-  .06 

the  push-over  limits  of  the 

5 

.06 

-  .08 

vehicles  for  which  we  compute 

6 

.08 

-  .10 

CCM) 

7 

.10 

-  .15 

3 

.15 

-  .20 

9 

.20 

-  .25 

10 

.25 

-  .50 

11 

.50 

-  1.00 

12 

1 

-  3 

13 

3 

-  5 

14 

5 

-  10 

15 

>  10 

Stem  Spacing  (a)  33  -  41  (4)  0  Mo  Data 

1-9  0  -  4.0  (by  0.5) 

10  4-5 

11  5-6 

12  6-8 

13  8-10 

14  10  -  15 

15  >  15 


Best  Available  Cod- 


460 


Bit 

Data  Element  Designation 

Vegetation  42  "  46 

Roughness  Factor 


#  of 

Sits  Code 


Value  Represented 


(5)  0 

1 
2 

3 

4 

5 
5 
7 

a 

9 

10-31 


0 

.1 

.2 

.3 

.4 

.5 

.6 

.7 

.8 

.9 

Not  Used 


Undergrowth 


47-46 


(2)  0  Mo  Data 

1  None 

2  Sparse 

3  Dense 


Tree  Crown  49-51  (3) 

Diameter  (meters) 

(Note:  Need  if  «*  intend  to  do  inter- 
visibility  or  Une-of-sight  products; 
otherwise  will  remain  teroed  out) 

Height  of  Lowest  52-54  (3) 

Branch  (meters) 

(Note:  Same  as  above) 


.  Surface  Material  Overlay 


Type 


55 


60 


(6) 


0 

1-7 


0 

1-7 


0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19-61 

62 

63 


No  Data 
Mot  Used 


No  Data 
Not  Used 


Mo  Data 

GW  -  Gravel,  well  graded 

GP  -  Gravel,  poorly  graded 

GM  -  Gravel,  silty 

GC  -! Gravel,  clayey 

SW  -  Sand,  well  graded 

SP  -Sand,  poorly  graded 

SM  -  Sand,  silty 

SC  -  Sand,  clayey 

ML  -  Silt 

CL  -  Clays 

0L  -  Organic  silts 

MH  -  Inorganic  silts 

CH  -  Fat  clays 

OH  -  Fat  organic  clays 

PT  -  Organic,  peat 

Snowfield/Glacier 

Rock  outcrops 

Evaporite 

Open  water 

Not  evaluated  (built-up  areas, 
etc) 


3 


Best  Available  Cop* 


ait 

#  of 

Data  Element  Designation 

aits 

Code 

Value  Represented 

Qualifier  SI  -  65 

(5) 

0 

No  Data 

1 

None 

2 

Eo ulder  field 

3 

Quarry,  mine,  diggings 

4 

3are  rock,  smooth 

5 

Lava  fow 

6 

Dunes 

7 

Loose 

3 

Karst 

9 

La ter 1 tic 

10 

Permafrost 

11 

Frequent  stone  or  rock  outcrops 

12 

Dissected 

13 

Metal/ore  slag  dump 

14 

Tailings,  waste  pile 

15 

Strip  mine 

16 

Rugged  bedrock 

17-31 

State  of  the  66  -  69 

Ground 

(4) 

0 

No  Data 

1 

Dry 

2 

Approximately  50?  saturated 

3 

Vet  (saturated) 

4-14 

0  -  1.0  (by  0.1 — fraction  of  soil 

moisture  in  top  one  half  meter 
at  time  of  measurement  or  CCM 
synthesization) 

15 

Depth  of  Surface  70  -  71 

(2) 

0 

No  Data 

Material  (meters) 

1 

0  -  0.5 

2 

3 

>0.5 

Surface  Roughness  72-75 

(4) 

0 

No  Data 

Factor  -  Medium  and 

1 

0 

Heavy  Tanks  (M-l  Abrams, 

2 

0.05 

M60,  and  T-72  Tanks,  etc) 

3 

0.1 

4 

0.2 

c 

\a 

1 

o 

5 

0.3 

6 

0.4 

7 

0.5 

8 

0.6 

(Note:  One  SR  table  will  be 

9 

0.7 

10 

0.75 

needed  for  each  vehicle  type 

11 

0.80 

for  which  a  CCM  map  is  to  be 

12 

0.85 

prepared) 

13 

0.90 

14 

0.95 

15 

1.00 

4 
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Best  Available  Co? 


463 


Best  Available  Co-" 


Bit 

#  of 

Data  Element 

Designation 

Bits 

Code 

7alue  Represented 

Height,  right 

105  -  107 

(3) 

0 

No  Data 

bank  (a) 

1 

•CO. 5 

2 

0.5  -  1.0 

3 

1.0  -  5.0 

4 

>  5.0 

5-7 

Height,  left 

108  -  110 

(3) 

0 

No  Data 

bank  (a) 

1 

C  0.5 

2 

0.5  -  1.0 

3 

1.0  -  5.0 

4 

>5.0 

5-7 

Slope,  right 

111  -  113 

(3) 

0 

No  Data 

bank  (5) 

1 

<30 

2 

30  -  45 

3 

45  -  60 

4 

>60 

5-7 

Slope,  left 

114  -  US 

(3) 

0 

No  Data 

bank  (J) 

1 

<30 

2 

30  -  45 

3 

45  -  60 

4 

>60 

5-7 

Water  velocity, 
average  (meters/ 

U7  -  U8 

(2) 

0 

No  Data 

1 

*2.5 

seconds) 

2 

>2.5 

3 

Net  Used 

Water  depth. 

U9  -  121 

(3) 

0 

No  Data 

average  (a) 

1 

<  0.8 

2 

0.8  -  1.6 

3 

1.6  -  2.4 

4 

>  2.4 

5-7 

Dense  Vegetation 

122 

(1) 

0 

No  Data 

Along  Stream  Banks 

1 

>50J  Segment  Lengti 

(Mots:  Normally  closely  spaced 
row  of  trees,  which  could  possibly 
hinder  stream  crossing  operations) 


Best  Available  C 


Data  Element 


att 

Designation 


*  of 

31 ts  Cade 


Value  Represented 


TVananartatlan  Overlay 
Type  123 


U6  (4)  0 

1 
2 
3 
U 

5 

6 

7 

8 
9 

10 

11-15 


Ho  Data 
Bridge  -  Road 
Bridge  -  Railroad 
Tunnel  -  Road 
Tunnel  -  Railroad 
Dual  Lane/Dlvided  Highway/ 
Expressway 
Highway/Road 
Railroad 
Airfield 
Inland  Waterway 
Lock 


Condition  1-27  -  129  (3)  ® 

(Note:  Need  part  of  this  field  2 

new,  will  need  the  rest  if  we  intend  3 

to  support  air  operations  or  on-route  4 
trafficabiiity  studies)  5 

7 


No  Data 

Good  Operational 

Fair  * 

Poor/ Deteriorated  " 

Damaged 

Destroyed 

Abandoned/Dismantled 
Under  construction 


Qualifier  130  —  ^32  (3) 

(Note:  Now  do  all  except  the 
grade  in  excess  of  3*  for 
railroads) 


0  No  Data 

1  Road  Constriction, <U  meters 

2  Grade  in  excess  of:  7'  for  roads 

or  35  for  railroads 

3  Sharp  curve  with  radius  <30  meters 

4  Perry  Site 

5  On  Route  Ford  Site 

6  Electrified  Line 

7 


Length  (meters)  133  -  138  M 

(Note:  For  this  Fort  Lewis-Takiaa 
prototype,  length  only  refers  to 
Bridges,  Tunnels,  Airfields,  and 
other  transportation  types  less  than 
100  meters  long.  All  lengths  greater 
than  100  meters  are  determined  by  the 
the  number  of  points  used  to  digitize 
the  length  of  the  feature  -  0.25  am 
(approx.  0.01  inches)  equals  12.5 
meters  on  the  ground  at  1:50,000 
scale) . 


0 

1 

2 

3 

u 

5 

5 

T 

8 

9 

10-31 


No  Data 
Unknown 
0-10 
10  -  20 
20  -  30 
30  -  40 
40  -  60 
60  -  30 
80  -  ICO 
>  100 
Not  Used 


Best  Available  Copy 


Data  Element 

Average  Width 
(meters) 


Bit 

Designation 


139  -  1«2 


(Kota:  For  this  Fort  Lewis- Yakima 
prototype,  width  refers  to  any 
transportation  type,  except  Railroads, 
less  than  50  meters  wide-  All  widths 
greater  than  50  meters  are  determined 
the  same  as  lengths  greater  than  100 
meters  -  see  above). 


Value  Represented 

So  Data 
Dnknown 
0  -  3 

3  -  4 

4  -  5 
5-7 
7-10 

10  -  20 
20  -  50 
>50 

Sot  Used 


Surface 


143  -  146 


(Note:  Need  part  of  this  field 
now,  will  need  the  rest  if  we 
intend  to  support  on-routa 
traff inability  studies) 


Highways  and/or  Roads: 


147  -  149 


0 

1 

2 

3 

4 

5 

6 
7 
3 
9 

10-15 


Ho  Data 

Paved 

Hard 

Loose/Gravel 
Onpaved 
Natural  Earth 
Grass 

Macadaa/Asphalt/3ituminous 

Concrete 

Stone/masonry/brick 
Not  CTsed 


No  Data 
All  Weather 
Fair/Dry  Weather 
Cart  Tracks 
Trails 


Railroads: 


150  -  152  (3) 


Passing  tracks, 
sidings  &  yards 
(meters) 


153  -  154 


No  Data 

Normal  Gauge,  single  track 
Normal  Gauge,  dual  track 
Normal  Gauge,  multiple  (2  or 
more)  tracks 

Narrow  Gauge,  single  track 
Narrow  Gauge,  multiple  track 
Broad  Gauge,  single  track 
Broad  Gauge,  multiple  track 

No  Data 

Passing  track  >280 
Siding  >280 
Yard  2 280 


Best  Available  Copy 


Data  Element 
Tunnels: 


Bit 

Designation 


Height  (meters)  155  -  157 


Value  Represented 


Ho  Data 
3  -  6 
6-3 
3-12 
>12 


Bridges: 


158  -  161 


(Note:  Need  if  we  intend  to  support 
engineer  or  on-route  trafficability 
studies;  otherwise  will  remain 
zeroed  out) 


Movement 


162  -  164 


0 

1 

2 

3 

4 

5 

6 
7 
3 
9 

10-15 


(Note:  Same  as  above) 


No  Data 

truss 

Girder 

Beam 

Slab 

Arch 

Suspension 
Floating 
Cable  stayed 
Cantilever 
Not  Used 

No  Data 
Fixed 
Not  Used 


Overhead 

Clearance 


Horizontal 

Clearance 


Onderbridge 

Clearance 


165  -  166  (2) 


167  -  168 


169  -  171 


No  Data 
Unknown 

Unlimited  clearance 
Possible  obstruction  to 
military  traffic 

No  Data 
Unknown 

Unlimited  Clearance 
Possible  obstruction  to 
military  traffic 

No  Data 
Untaown 
0-5 
5-10 
10  -  50 
>50 

Not  Used 


Best  Available 


Bit 

#  of 

Data  SI • nent 

Designation 

Bits 

Code 

7alue  Represented 

Bypass 

172  -  173 

(2) 

0 

Ho  Data 

Conditions 

1 

Easy 

(Potential  within 

2km) 

2 

Difficult 

3 

Impossible 

Construction 

174  -  176 

(3) 

0 

Ho  Data 

Material 

1 

Other 

2 

Wood 

3 

Stone/masonry/brick 

4 

Steel 

5 

Concrete 

6 

Reinforced  concrete 

7 

Prestressed  concrete 

Classification 

177  -  180 

(4) 

0 

Ho  Data 

(one-way  wheeled) 

1 

50 

(metric  tons) 

2-9 

10-15 

Hot  Used 

Classification 

181  -  184 

(4) 

0 

Ho  Data 

(one-way  tracked) 

1-7 

0-60  (by  10) 

(metric  tons) 

8 

61  -  100 

9 

>100 

10-15 

Hot  Used 

Reliability  cf 

185  -  186 

(2) 

0 

Ho  Data 

Bridge  Classification 

1 

Unknown 

2 

Known 

3 

Estimated 

Specs  (number) 

187  -  190 

(4) 

0 

Ho  Data 

1 

Unknown 

2 

1 

3 

2 

4 

3 

5-8 

4  -  11  (by  2's) 

9 

212 

10-15 

Span  Length 

191  -  193 

(3) 

0 

Ho  Data 

(meters) 

1 

Unknown 

2 

C25 

3 

25-50 

4 

50  -  100 

5  >100 

6-7 


10 

B©st  Avst’lsbl©  Cor 


Data  Element 


ait 

Designation 


Value  Reoresented 


71.  Obstacles  Overlap 


19*  -  196  (3) 


(Note:  Linear  obstacles  are 
defined  as  any  hir.derance  to 
movement  which  is  greater  than 
1.5  meters  high,  has  a  453  or 
greater  slope,  and  is  at  least 
250  meters  long.  Areal  obstacles 
are  defined  as  any  area  which  is 
so  characterized  as  to  severely 
restrict,  stop,  or  otherwise  make 
movement  impractical.  This  over¬ 
lay  depicts  land  obstacles  only- 
user  is  referred  to  Surface 
Drainage  Overlay  for  hydrologic 
obstacles.] 


Height  (m) 


197  -  199  (35 


Ho  Data 

Road  and  RR  cuts  and  fills 

Natural  linear  obstacles  (escarp¬ 
ments,  dikes,  cliffs,  etc.) 

Walls  and/or  fences  (hedgerows, 
rock  and  wire  fendes  and  walls, 
retaining  walls,  etc.) 

Other  man-made  linear  obstacles 
(dikes,  moats,  embankments, 
etc) 

Military  obstacles  (antitank 
ditches,  airfield  and/or  road 
craters,  blown  bridges,  debris 
choked  valleys  and/or  towns. 
Impact  areas,  minefields,  road¬ 
blocks,  trenches,  wire  entangle¬ 
ments,  etc.) 

Man-made  areal  obstacles  (mining 
operations — pits,  quarries, 
strip  mines,  etc. — terraced 
hills  and/or  paddies  (wet  and 
dry)) 

Natural  areal  obstacles  (craters, 
dissected  land,  talus  piles, 
depressions,  sink  holes,  open 
water,  etc.) 

No  Data 
>1.5 

1.5  -  5 
5-10 
10  -  20 
20  -  35 
>35 


The  Prototype  Fort  Lewis-Takiaa  Training  Area  TTADB  matrixed  format  ends  with 
bit  199.  The  full  proposed  TTADB  includes  224  bits  and  includes  additional 
overlays  for: 

(1)  Aerial  Obstructions. 

(2)  Special  Features/Preduct  Synthesis  (customer  related  overlays/standard¬ 
ized  and  future. COM,  Concealment,  etc.  overlays). 

• 

(3)  Text  Data  (two)  -  for  such  information  as  bridge  tables,  climatic 
data,  hydrologic  flow  graphs,  names,  generalized  descriptors,  etc.  On  account 
of  the  limitations  mentioned  on  page  1,  rather  than  carry  25  bits  packed  with 
•/.croes  at  the  end  of  each  data  point,  it  was  decided  to  omit  these  fields  from 
the  Fort  Lewis-Takira  Training  Area  TTADB. 


Best  Available  Copy 
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SUBJECT:  Simple  Analysis  Methods  by  Which  to  Discriminate  Among  Terrain 
Compartments* 

The  purpose  of  this  paper  is  to  simply  describe  what  data  may  be  used  and,  due 
to  whether  this  data  is  expressed  as  discrete  or  continuous  variables,  what 
analytic  tools  might  be  brought  to  bear  in  the  determination  of  whether  the 
terrain  compartments  are  equal  or  whether  there  are  differences  between  them. 
These  analyses  may  lend  themselves  to  answer  such  questions  as:  whether  the 
Natick  plus  Veg-Urb  classifications  adequately  and/or  accurately  describe  the 
various  compartments;  what  variables  show  the  greatest  discriminative  ability, 
acting  alone,  between  compartments;  are  there  interaction  effects  between  the 
levels  of  two  variables;  etc.  These  procedures  are  labeled  simple  analyses  in 
that  they  are  precursors  to  more  involved  techniques  which  seek  to  uncover  the 
actions  and  interactions  of  several  variables  as  they  are  expressed  in  the 
compartments.  The  techniques  which  shall  be  used  on  "continuous"  variables 
(those  for  which  at  least  the  mean,  standard  deviation,  and  n  are  available) 
consist  of  one  way  analyses  of  variance  (ANOVA),  and  following  a  significant 
F,  the  determination  of  which  means  are  equal,  which  are  unequal  through  the 
use  of  such  a  posteriori  methods  as  the  least  significant  difference  (lsd) 
test  or  the  Student-Newman-Keul s  multi  range  procedure.^ 


One  Way  ANOVA  for  13  Compartments 

Source 

SS 

df 

MS  F 

MS  expected  value 

Compartments 

ssc 

12 

SSc/df  MSc/MSe 

N^-£nj2  +  a2 

t  e 

N(12) 

Error 

sse 

N-13 

SSo/df 

E(MSerror)=a2 

e 

Total 

sst 

N-l 

In  several  occasions,  where  there  are  two  levels  of  a  variable  (such  as 
in-view/out-of-view)  a  two  way  ANOVA  should  be  appropriate,  with  the 
determination  of  the  significance  of  the  interaction  between  the  two  variable 
levels  and  the  compartments. 


1  This  paper  was  done  not  only  to  outline  the  ways  to  analyse  the  terrain  data , 
hut  aleo  to  serve  as  a  refresher  in  the  mechanic's  of  the  various  techniques 
to  those  who  may  he  a  little  "rusty". 

2 References  for  the  ANOVA  and  a  posteriori  contrasts  are  Steel ,  R.G.D.  <S  J.H. 
Torrie;  Principles  and  Procedures  of  Statistics;  New  York ,  McGraw-Hill;  1960 
and  Winer ,  B.J.;  Statistical  Principles  in  Experimental  Design,  2nd  Edition , 
New  York,  McGraw-Hill;  1  971  . 
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Two1  Way  aNOVA  (equal  cell  ITzeJ  tor  l3  Compartments 


Source 

ss 

df 

ms 

f 

ms  expected  value 

Compartments 

SSC 

12 

ssc/dfc 

msc/mse 

2n<j2  +  o2 
c  e 

IV/OOV 

ss-f 

1 

ss(/dfi 

msi/mse 

13  n2  +  a2 

I  e 

Comp  x  IV/OOV 

SSX 

12 

ssx/dfx 

ms  x/mse 

no2 ,  +  a2 
cl  e 

Error 

SSe 

13  *  (n-1) 

sse/dfe 

o2 

e 

ToTal - rFJ^rTPT 


An  aposterlorl  contrast  test  Is  a  systematic  procedure  for  comparing  all 
possible  pairs  of  group  means.  The  groups  are  divided  Into  homogeneous 
subsets  where  the  difference  In  the  means  of  any  two  groups  In  a  subset  Is 
not  significant  at  some  prescribed  significance  level  (alpha).  The 
procedure  is  based  on  the  test: 

<  R(a1pha,  g,f)*ST 

Where  R(alpha,g,f)  Is  a  range  based  on  a  significance  level  (alpha),  the 
number  of  groups  n  the  subset  (g),  and  the  degrees  of  freedom  (f)  In  the 
between-groups  sum  of  squares  (error  degrees  of  freedom),  s  Is  the  standard 
error  In  the  combined  subset,  and  Is  equal  to  (MSerror)*/*,  * 

The  least  significant  difference  test  (Isd)  Is  essentially  a  Student's  t  test 
between  group  means.  It  Is  usually  not  recommended  since  as  the  number  of 
groups  Increases,  so  does  the  experiment  wise  error  rate.  However,  with  the 
prior  determination  of  a  significant  F,  which  puts  an  upper  bound  on  the 
experiment  wise  error  rate,  the  Isd  procedure  Is  considered  are  appropriate 
liberal  test.  Isd  is  also  exact  fo«"  unequal  group  sizes.  The  Isd  Is  computed 
asToT lows:  lsd«t  stf,  where  t  Is  the  Student's  t  for  the  chosen  significance 

level  and  error  degrees  of  freedom.  Sd»(MSePror  +  J-,))l/2  where  rl  and  rj 

r  1  rj 

are  the  sample  sizes  of  the  two  means  being  compared.  If  |x1-xj|>lsd,  then 
the  two  means  are  considered  significantly  different.  The  alpha  level  of 
significance  may  be  modified  for  the  Isd  procedure,  and  take  care  of  the 
expanding  experiment  wise  error  rate  by  choosing  a  lower  alpha. 

The  Student-Newman-Keuls  (SNK)  test  attempts  to  avoid  the  expanding  error  rate 
problem  In  the  Isd  by  using  a  different  range  value  for  subsets  of  different 
sizes  (the  larger  the  number  of  groups  In  a  subset  the  larger  the  difference 
In  the  means  must  be  in  order  to  be  declared  significant).  The  SNK  test  also 


...JLjJS 


471 


uses  the  concept  of  a  special  protection  level  rather  than  a  significance 
level:  the  probability  of  finding  a  significant  difference,  given  that  two 
groups  are  In  fact  equal,  Is  less  than  or  equal  the  specified  significance 
level.  SNK  holds  the  experiment  wise  error  rate  to  alpha  for  each  stage  of 
the  testing  procedure.  This  procedure  Is  more  conservative  than  the  lsd 
procedure  (a  greater  tendency  to  find  means  not  significant,  than  the  lsd), 
however  the  validity  of  this  procedure  for  handling  unequal  group  sizes  has 
not  been  verified.  The  following  Is  offered  as  an  example  of  the  one-way 
ANOVA  followed  by  a  posteriori  comparisons.  This  is  a  comparison  of  four 
terrain  compartments  for  first  observation  ranges: 


1 

2 

3 

4 

N 

86 

282 

108  1 

142 

T 

3240 

3298 

2404 

2788,3 

Sd 

988.62 

:■  11878.2 

1384.  18 

1506.84 

One  Way  ANOVA 

df 

ms 

■  1 

Compartments 

3 

24854429.42 

Error 

614 

1635639.407 

Total 

617 

f 

15.2  s1g<,001 


A  posteriori  comparisons  - 

difference 


Compartment 

X 

4 

1 

2 

3 

2404 

383.7 

836 

894 

4 

2788.3 

451.7 

509.7 

1 

3240 

56 

2 

3298 

lsd  t. 99,Gi7"2. 33  one  tall,  lsd-t(MSe(*  +  1  ))i/2 

Ti  n 

mean  384  lsd-380.5  SI g,  4>3  mean  481  1  sd-407. 1 68.'. SI g  1>4 
mean  381  lsd-430.66.*.  SI g,  1>3  mean  482  lsd-306.63.\S1g  2>4 
mean  382  lsd-337.2.%  SI g,  2>3  mean  182  lsd-367. 07.*.NS  1-2 


The  results  of  the  lsd  test  can  be  summarized  by  saying: 

The  mean  of  compartment  3  Is  less  than  the  mean  of  compartment  4,  which  Is 
less  than  the  means  of  compartments  1  and  2,  which  are  equal;  or  3<4<l-2. 


472 


iV 

U 

I 

i 


The  SNK  test  Is  performed  by  first  looking  up  In  a  table  the  multipliers 
associated  with  the  number  of  means  across  which  the  test  Is  to  be  made.  For 
Instance,  In  the  above  table  of  ordered  means  a  test  of  3  and  4  would  be 
across  two  means,  3  and  1  across  three  means,  3  and  2  across  four  means.  If 
one  looks  In  a  table  of  the  Studentlzed  range  statistic3  for  alpha  of  .01,  df 
of  m  the  multiplier  q  for  2  is  3.64,  for  3  is  4.12,  for  4  Is  4.40.  The 
critical  value  for  the  difference  between  two  means  Is 

q  ^MSerror^/2 

where  N  Is  the  harmonic  mean,  k  Is  number  of  compartments 


n- _ k _  ■> _ 4 _ -127.08 

(Vn  l)  +  l/n2  +•"•“•+■(  i /n|()  ^  ^  /a  •  a  j + ( *  /i  <>•)♦(*  /i  42 ) 


so 


^iar-ror) 


V2  -  113.45 


and  the  critical  value  between 


2  means  Is  412.95,  between  3  Is  467.41,  between  4  Is  499.18.  By  examining  the 
mean  difference  table  above,  one  may  see  that: 


t4-t3"383*7  <  412*95  3*4  not  different 

i  Tl-*3»836  >  467.41  .♦.  significant  1>3 

t  T2-T3"894  >  499,18  .*.  significant  2>3 

i  Tl-*4«451.7  >  412.95  .*.  significant  1>4 

|  T2-T4-509.7  >  467.41  significant  2>4 

l  r2.rr 58  <  412.95  A  1  &2  not  different 

i  The  results  of  the  SNK  may  be  summarized  as  follows:  3*4<1«2. 

i 

'  This  also  serves  as  an  example  of  the  difference  between  the  "liberal"  lsd  and 

the  "conservative"  SNK.  Take  the  difference  between  mean  3  and  mean  4;  lsd 
f  found  these  means  as  significantly  different,  even  though  close  to  the 

i  critical  value,  whereas  SNK  did  not  reject  the  null  hypothesis  of  no 

■>  difference  between  them.  It  depends  on  the  audience,  but  I  would  tend  to  go 

;  with  the  lsd  results,  due  to  the  prior  significant  F  and  the  associated 

|  Increase  in  power  the  lsd  allows. 


Viner,  opoit  pp  870-871 . 
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For  the  two  way  analysis  of  variance,  the  same  procedures  apply;  to  test 
between  two  means,  use  a  t-test  or  Isd  test,  using  the  MS  within  cell  and  Its 
associated  df  to  determine  the  criterion  value  of  t.  In  making  all  possible 
tests  between  ordered  means,  the  SNK  procedure  Is  to  be  used,  using  the 
harmonic  mean  of  the  cell  sample  sizes  for  n.  It  Is  also  possible  to  test 
the  significance  of  the  simple  effects  of  one  factor  over  one  level  of  a 
second  factor,  especially  useful  If  there  Is  Interaction  present  between  the 
two  factors.  For  example,  the  sample  effects  of  the  four  means  of  a  factor  B 
over  the  second  level  of  A,  Az,  would  be 


SSbfora,  -Nh  [(5*1 ,2  )*+(*2 ,2 )2+(Fa  ,2 )2+(Fn  ,2  )2 -(£  bj,2  )2  ] 

4 


M$bforaa 


SSbfora2 
— 3 - 


MS  b  for  a2  (the  MS  within  cell  Is  from  the  2-way  ANOVA) 

F  ■  within  cell 

The  df  for  this  F  are  3  and  the  df  within  cell. 


At  this  time,  the  following  continuous  variables  are  available  for  the  terrain 
compartments; 

a.  Thickness  and  separation  measures  for  vegetation,  urban  features,  and 
the  combination  of  vegetation  and  urban  features. 

b.  In-view  and  out-of-view  segment  lengths. 

c.  First  opening  range. 

d.  Number  of  In-view  and  out-of-view  segments  per  route. 

It  may  be  necessary  to  transform  some  of  these  variables  so  that  they  more 
nearly  represent  a  normal  distribution.  This  will  be  determined  as  the  data 
becomes  available  which  will  allow  a  determination  of  the  underlying 
distribution.  Battle  outcome  may  also  be  considered  a  contlnous  variable  when 
sufficient  replications  are  performed;  Battle  outcome  will  then  be  handled  In 
a  repeated  measures  ANOVA  design. 

Discrete  variables  are  those  In  which  the  data  Is  broken  Into  categories, 
which  have  a  frequency  of  occurrence  attached  (bean  counting)4.  When  the 
categories  are  considered  nominal  (just  names-no  relation  between  them) 


^Rafaranoaa  for  ncmpamniotrio  amlyata  prooaduraa  Inoluda  brail  ay,  J.V.j 
Diatrtbution-Fraa  Statiatioal  Taata:  Englauood  Cliffa,  N,J.t  Prantioa-Hall, 

1  088  and  Siagal,  81  RempammatrisstaHatioa :  For  tha  BahaHoml  Soianaaa ;  Ran 
York ,  bbGmto-Hlli  1  968. 
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measures  of  association  such  as  Chi-square,  or  Mueller's  lambda  are 
appropriate.  However,  by  ordering  the  terrain  compartments  by  some  means 
(such  as  the  Natick  system)  then  It  Is  possible  to  use  measures  of  association 
which  depend  on  ordinal  variables  (ordered,  but  not  assuming  a  constant 
separation  between  variables)  such  as  tau,  Gamma,  or  Somer's  D. 

The  Chi-square  Is  a  test  which  measures  whether  a  systematic  relationship 
exists  between  two  variables  (that  Is,  whether  the  variables  are  dependent  or 
Independent).  This  Is  done  by  computing  the  cell  frequencies  which  would  be 
expected  If  no  relationship  Is  present  between  the  variables  given  the 
existing  row  S  column  totals  (marginals).  The  expected  cell  frequencies  are 
then  compared  to  the  actual  values  found  In  the  contingency  table  according 
to  the  following  formula: 


where  f0  equals  the  observed  frequency  in  each  cell,  and  fe  equals  the 
expected  frequency  calculated  as 


fe 

where  cj  Is  the  frequency  In  a  respective  column  marginal,  n  1*  the  frequency 
In  a  respective  row  marginal,  and  N  stands  for  the  total  number  of  valid 
cases.  Small  values  of  V  indicate  the  absence  of  a  relationship,  or 
statistical  Independence.  Conversely,  a  large  T  Implies  that  a  systematic 
relationship  of  some  sort  exists  between  the  variables. 

Mueller's  asymmetric  lambda  measures  the  percentage  of  Improvement  In  the 
ability  to  predict  the  value  of  one  variable  once  the  value  of  the  other 
variable  Is  known.  This  Is  based  on  the  assumption  that  the  best  strategy 
for  prediction  Is  to  select  the  category  with  the  most  cases  (modal  category), 
since  this  will  minimize  the  number  of  wrong  guesses.  This  concept  Is 
called  the  proportional  reduction  In  error.  The  formula  for  lambda  Is 


fcmaxfjk  -  Maxf.k 
*b  *  n  ..  maxf.k 

where  maxfj|<  represents  the  sum  of  the  maximum  values  of  the  cell 
frequencies  In  each  column,  and  maxf.k  represents  the  maximum  value  of  the  row 
totals. 
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Tau  b,  Gamma,  and  Somer's  n  are  measures  of  association  between  two 
ordinal-level  variables,  and  are  built  upon  a  common  basis.  They  use  the 
Information  about  the  ordering  of  categories  of  variables  by  considering  every 
possible  pair  of  cases  In  the  table.  Each  pair  Is  checked  to  see  If  their 
relative  ordering  on  the  first  variable  Is  the  same  (concordant)  as  their 
relative  ordering  on  the  second  variable,  or  If  the  ordering  Is  reversed 
(discordant). 

The  first  step  Is  to  compute  the  number  of  concordant  pairs  (P)  and  discordant 
pairs  (Q).  If  P  Is  larger  than  Q,  this  means  that  there  Is  a  preponderance  of 
pairs  ordered  In  the  same  direction  on  both  variables,  and  the  statistic  will 
be  positive.  Conversely,  a  larger  Q  will  result  In  a  negative  statistic. 

This  positive  or  negative  association  Is  also  referred  to  as  correlation. 

Tau  b  •  P-0 

l>  /a  (n  (n  - TFFfi  ( T  i  - 1 ) )  ‘  /z ( N  ( N- 1 J.  ilfi  - 1 ) )  /2 


where  Ti  Is  the  number  of  ties  on  row  variables,  T2  Is  the  number  of  ties  on 
the  column  variables. 

Gamma  makes  no  adjustment  for  ties  or  table  size,  and  Is  simply 

Gamma  ■  P-Q 
T+Q 


A  variation  on  Gamma,  which  accounts  for  ties  but  not  table  size  Is  Somer's 
Somer's  0  (symmetyrlc)  ■  P-Q 

WFWi+Ta) 


These  measures  differ  basically  on  the  manner  In  which  ties  are  handled,  and 
are  basically  used  to  give  an  Indication  of  the  type  and  relative  strength  of 
the  association  between  two  variables. 


A  Friedman  2  way  ANOVA  by  ranks,  may  be  appropriate  depending  on  the 
appearance  of  the  data  and  the  number  of  tied  values.  The  comparison  of 
two  distributions  for  equality  may  be  performed  by  using  the  Kolmogorov- 
Smlrnov  two-sample  test.  These  procedures  are  appropriate  for  comparing 
between  terrain  compartments  for  modal  hill  height,  and  PLOS  vs.  range  when 
range  Is  expressed  as  range  bands. 


The  Friedman  two-way  ANOVA  would  be  used  to  test,  for  example,  that  each 
terrain  compartment  was  ranked  the  same  In  each  range  band.  Given  k 
compartments  In  columns  and  N  rangebands  In  rows,  rank  the  scores  In  each  row 
from  1  to  k.  Determine  the  sum  of  ranks  In  each  column:  Rj,  and  compute 


*2 

r 


12 

RFflTfi ) 


2  (RJ )2-3N(k+l) 
j-1 
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wnere  M«number  of  rows 
k»number  of  columns 
Rj-sum  of  ranks  in  column; 

K2  is  approximated  by  *2  with  df-k- 1.  If  the  value  of  Is  equal  to  or 
r  r 

larger  than  a  tabled  value  of  ,  the  Implication  Is  that  the  sum  of  the 

k-1 

ranks  for  the  various  columns  differ  significantly.  The  Kolmogorov-Sml rnov 
two-sample  test  can  be  used  as  a  suitable  a  posteriori  comparison  of 
distributions  after  the  Friedman  test.  The  procedure  for  this  test  would  be 
to  compute  the  cumulative  distribution  of  each  of  the  two  compartments  under 
test,  then  locate  the  range  band  where  the  difference  between  the  cumulative 
percentage  In  one  compartment  Is  maximized  with  relation  to  the  second 
coinoartment.  This  difference  0  Is  computed  as 
e*i  c£, 

d  «  M*x[-—  -  ---J  for  a  one-tall  test. 
nl  n2 

where  m  and  nz  are  the  sample  sizes  of  compartment  1  and  compartment  ?., 
respectively.  The  statistic  Is  computed,  for  large  n,  as 

*2-“!  ‘K-H71 

which  Is  approximated  by  the*2  distribution  with  df-2.  If  this  statistic  Is 
found  significant,  then  the  two  distributions  are  different.  The  rationale 
behind  this  test  Is  that  If  the  two  samples  were  distributed  equally,  their 
cumulative  distributions  would  not  be  very  different. 

Some  of  the  variables  encountered  have  a  single  value  per  terrain  compartment. 
These  may  be  used,  given  the  terrain  compartments  are  ordered,  by  using  a 
technique  such  as  the  Spearman  or  Kendall  rank-order  correlation.  A 
significant  positive  correlation  would  Indicate  that  the  compartments  were 
properly  ordered  with  respect  to  the  observed  variables.  Single  value  data 
that  would  be  analyzed  with  this  technique  would  be  maximum  hill  height,  the 
number  of  positive  features  per  kilometer,  the  total  number  of  positive 
features,  and  battle  outcome  results  If  not  replicated. 

The  Kendall5  and  Spearman  rank  correlations  are  correlations  based  upon  the 
ordinal  ranks  of  the  observations  of  two  variables,  and  not  on  their  observed 
value,  let  n,  rz,  ...rn  represent  the  ranks  of  tha  values  of  one  variable, 
and  si,  sz,  ...sn  the  representative  ranks  of  the  second  variable. 

Correlation  can  be  tested  by  arranging  the  n  units  In  Increasing  order  on  the 
r  variable,  and  testing  the  resulting  order  of  the  s  variable  for  randomness. 


fire/«ral«o  to  Kendall,  M»Gt,  and  At  Stuarts  7k 0  Advanood  7k  torn  of 
StaH$Ho9t  Sol  t,  Srd  Ed:  Jfew  York,  Hafntr,  Ufa,  pp4$f-B05.  *  ' 
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If  the  two  variates  are  Independent  (so  that  there  Is  no  correlation),  the 
resulting  sequence  of  s  observations  Is  equally  likely  to  be  any  of  the  ni 
possible  permutations  of  the  n  S's.  However,  If  the  two  variables  are 
linearly  (or  even  "monotonlcally")  correlated,  the  S-observatlons  should  tend 
to  form  an  Increasing  or  decreaslg  sequence,  and  any  statistic  that  reflects 
this  Increase  or  decrease  can  be  used  to  test  for  correlation.  (The 
advantage  of  the  Kendall  over  the  Spearman  Is  that  the  Kendall  can  be 
generalized  to  a  partial  correltlon  coefficient.)  The  Kendall  rank 
correlation  Tb  1$  defined  as 

Tb-  P-Q 


N(n-1)  where 

P-twIce  the  number  of  pairs  of  rankings  such  that  both  rj>rt  (the  Increasing 
order)  and  Sj>ss  (also  Increasing  order,  thereby  agreement  In  rank  order 
direction). 

?  -twice  the  number  of  pairs  of  rankings  such  that  rj>m  and  $j<Si 
disagreement  In  rank  order  due  to  the  “Inversion"  of  s  from  the  “natural" 
ascending  order).  When  rankings  are  tied  In  either  r  or  s  the  formula  for  tb 
Is 

Where  T-t^MT.,  ,)  and  T,  Is  the  number  of  observations  tied  with  a  single 
value,  me  sunt  £*  Is  over  all  distinct  values  for  which  a  tie  exists.  Ti 
Is  the  total  for  the  first  variable  r,  Ts  for  s. 

The  Spearman  rank  correlation  Is  defined  as 

6i(n  -  s*)2 

r‘  ■  1  ’ 

when  rankings  are  tied,  rs  Is  modified  to 
Ts  *•  A+U-D 

where  A  -  (N3-N-Ti)/12 
B  -  (N»-N-Ta)/12 
0  •  c(rj-sj)* 

and  T  ■  t*  (tJ-Ti),  defined  as  above. 


As  an  example  of  these  procedures,  consider  the  following  set  of  data* 


Group 

Attribute  A 

Attribute  B 

rJ-*j 

(rj-sj)* 

A 

2 

li 

3 

-1 

1 

B 

6 

4 

2 

4 

C 

5 

2 

3 

9 

D 

1 

1 

0 

0 

E 

10 

8 

2 

4 

F 

9 

11 

-2 

4 

G 

8 

10 

-2 

4 

H 

3 

6 

•3 

9 

I 

4 

7 

*3 

9 

J 

12 

12 

0 

0 

K 

7 

5 

2 

4 

L 

11 

9 

2 

4 

r? 5? 


for  tha.Soearman  rank  correlation,  this  would  turn  out  to  be 
n 

rs  -  1  -  6  *  {ri-Sj)»  ■  1-6(52)  “.82 

JZ2 -  1716 

N3-N 


when  H  Is  10  or  larger,  the  significance  of  an  obtained  rs  under  the  null 
hypothesis  may  be  tested  by  t  ■  and  compare  with  student's  t  for 

df  ■  N-2. 

For  this  example 

"  •82(y^4g|y5r)11/2  >  4.53  >tio,/>05  for  a  one  tall  test. 


8frcm  Sitgtl  pp  SOS 


The  corresponding  Kendall  correlation  Is  computed  as  follows: 


Group 
Att  A(r-j) 

Att  B(Sj) 

As  an  easy  way  of  computing  P  and  Q,  V2P  and  1/2Q  Is  measured  as  the  number 
of  ranks  of  s;  to  the  right  of  each  Individual  si  that  are  either  greater 
than  or  less  than  the  subject  rank  (or  1/2  Pj  ana  */2  Qj,  respectively). 
Therefore 


IP 

~T~ 

"P 

HP 

PT 

-P 

~r 

nr 

nr 

rp 

PI 

2 

3 

4 

5 

& 

7 

5 

§ 

id 

11 

12 

p 

5 

2 

6 

7 

3 

4 

10 

11 

3 

§ 

12 

l/2(P-Q)-(ll-0)+(7-3)+(9-0)+(6-2)+(5-2)+ 
(6-0) +(5-0) +(2-2) +(1-2) +(2-0) +(1-0) "44, 


(The  attribute  rank  farthest  to  the  left  Is  1.  This  rank  has  11  ranks  to  the 
right  which  are  larger,  0  which  are  smaller  so  1/2  (P1-Q1 )»(ll-0),  and  so  on). 
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When  N  Is  larger  than  10,  t  may  be  considered  normally  distributed  with 


Mean*MT=0  and  standard  devlatlon-oT 


l/2 


SO  2*  T-Mt* 
oT 


In + 5 


Va 


Is  approximately  normally  distributed  with  2ero  mean 
the  significance  of  z  may  be  determines  by  reference 
table.  For  the  example,  the  test  of  whether  the  two 
can  be  computed  as: 


and  unit  variance.  Thus 
to  an  appropriate  z 
variables  are  associated 


.67 

1/2 


3.03 


by  reference  to  a  z  table  It  Is  found  that  z  >3.03  has  the  probability  of 
occurance  under  Ho:  no  association,  of  p«.001I,  this  Ho  Is  rejected,  and  It  Is 
concluded  that  the  two  variables  are  associated.  You  will  note  that  the 
Spearman  and  Kendall  procedures  produce  different  coefficients  of  correlation 
when  both  were  computed  from  the  same  pair  of  rankings.  These 
examples  Illustrate  the  fact  that  t  and  rs  have  different  underlying  scales, 
and  so  numerically  are  not  directly  comparable  to  each  other.  However,  both 
coefficients  utilize  the  same  amount  of  Information,  and  thus  both  have  the 
same  power  to  detect  the  existence  of  association  In  the  population.  When 
used  on  data  to  which  the  Pearson  r  is  properly  applicable,  both  t  and  r$  have 
efficiency  of  91  percent. 
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As  was  Initially  stated,  these  methods  are  Initial  ones,  meant  to  see  how 
similar  and/or  dissimilar  the  terrain  compartments  are.  Higher  order  analyses 
planned  are:  factor  analyses  to  see  how  the  several  variables  relate; 
multiple  linear  and  nonlinear  regression,  to  determine  the  effects  of 
variables  on  dependent  variables  of  Interest,  such  as  segment  length,  or 
observer-target  distance;  time  series  analyses  to  determine  the  terrain 
effects  on  variables  of  Interest  as  time  progresses;  and  analyzing  differences 
In  distributions  as  to  their  possible  underlying  causes.  These  higher  level 
analyses  will  be  undertaken  as  the  data  becomes  available  and  programs  are 
operational. 

Also  to  be  undertaken  In  the  analysis  of  terrain  Is  whether  Items  of  Interest 
(such  as  line  of  sight,  or  battle  outcome)  vary  over  time,  or  distance  from 
observer  to  target,  and  whether  this  variation  can  be  expressed  by  an  equation 
which  Is  distinct  for  the  separate  types  of  terrain  as  Identified  by  the 
(modified)  Natick  system.  Variables  which  may  be  analyzable  In  this 
determination  may  be: 

a.  Percentage  of  Line  of  Sight  (Pl0S)  -  for  each  unit  of  time  t,  for 
observer  array  0,  target  array  T,  the  proportion  of  targets  to  which 
Intervlslblllty  exists,  denoted  P  (t). 

b.  Correlation  (or  Coherence);  -  for  each  unit  of  time  t,  for  0  and  T, 
the  correlation  of  several  observers  seeing  several  targets,  denoted  p(t). 

c.  Percentage  of  Llne-of-SIght  and  correlation,  as  above,  varying  as  the 
distance  L  from  0  and  T  (In  the  case  of  correlation  this  distance  may  need  to 
be  to  the  centroid  of  the  array  T),  denoted  P(i)  and  p(j,). 

d.  Battle  outcome  -  In  terms  of  targets  killed  by  observers  killed  per 
unit  time  t,  denoted  k(t). 


' In  the  tempi*  oat*  of  two  dbtarvare  rating  a  group  of  targets,  oorralation  it 
d*fin*d  at  p  m  P(AB)-P(A)P( B) 

mm-r.wiP(B)(i-p~iB) 

Dr,  Wilbur  Payn*  d*fin*t  ooh*reno«  at  th*  ttat*  Ji an  p>0,7. 


wpsvjmmQ  \  jiniMj.^ij  jnpp$s!TC . 


In  several  recent  articles8  there  has  been  described  the  potential  for 
describing  terrain  with  an  equation,  and  that  the  spectral  density  of 
fluctuations  In  terrain  (as  well  as  many  other  physical  quantities  such  as 
music  and  speech)  vary  as  1/f,  where  f  Is  the  frequency.  This  1/f  behavior 
Implies  some  correlation  In  fluctuating  quantities,  such  as  PLOS,  overall 
times  corresponding  to  the  frequency  range  for  which  the  spectral  density  Is 
1/f,  such  as  may  be  found  In  a  homogeneous  terrain  compartment. 

The  spectral  density  Sp(f)  of  a  quantity  P(t)  fluctuating  with  time  t  Is  a 
measure  of  the  mean  squared  variation  <P2>  over  the  terrain  compartment.  A 
second  characterization  of  the  average  behavior  of  V(t)  Is  the  auto 
correlation  function,  <P(t)P(t+T)>,  which  Is  a  measure  of  how  the  fluctuating 
quantities  at  times  t  and  t*T  are  related.  For  a  stationary  process 
<P(tlP(t+T)>  Is  Independent  of  t  and  depends  only  on  the  time  difference  t. 
$p(f)  and  <P(t)Pt+T)>  are  not  Independent,  but  are  related  by  the  Wlener- 
Khlntchlne  relations9. 

<P(t)P(t+T)>  -jSp(f)  cos(2nfT)df 
and  Sp(f )»4/ <V(t)V(t+f)>  cos{2nfT)dT 

The  methods  for  determining  the  appropriate  f,  spectral  densities,  and  series 
analysis  have  not  been  fully  worked  out  at  this  time,  but  will  probably 
Involve  the  use  of  the  Spectral  Anaysls  and  the  Box-Jenklns  Analysis  routines 
contained  In  the  1981  release  of  the  BMDP  stastlstlcal  package1^. 


PAUL  DEASON 


8 Cunningham ,  E.P.;  "Single-Parameter  Terrain  Classification  for  Terrain 
Following";  J.  Aircraft ,  1980 ,  pp  909-974, 

Voss,  RF,  and  John  Clarke i  "1/f  noisd'  in  music:  Music  from  1/f  noise " 
Aaouet.SooAm  63(1),  Jan  78,  268-263*  Gardner,  Mi  "White  and  brown  msic, 
fractal  ourvee,  and  one-over-f  fluo tuations"  Mathematical  Games  Section  of 
Scientific  American ,  1  978, 

^ Reif ,  F,  Fundamentals  of  Statistical  and  Thermal  ftiusioej  New  York, 

Ms Graw-Hi ll,  1  965,  pp  686-687, 

10  Biomedical  Computing  Programs ,  Health  Sciences  Computing  Facility, 
University  of  California  at  Los  Angeles,  1981, 
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ERROR  PROPAGATION  IN  PHYSICAL  MODELS 


Jerry  Thomas  and  J.  Richard  Moore 
US  Army  Ballistic  Research  Laboratory 


ABSTRACT.  Let  Y  ■  g(X)  be  an  equation  which  describes  a  physical  process. 
If  the  argument  of  this  function  is  subject  to  error,  then  Y  will  reflect  this 
error.  The  transfer  of  error  in  X  to  error  in  Y  through  g(X)  is  commonly 
called  error  propagation.  This  paper  gives  a  collection  of  procedures  for 
studying  commonly  used  error  propagation  equations  and  imp coving  them  when 
necessary. 

1.  INTRODUCTION .  A  physical  model  is  defined  for  tho  purpose  of  this 
discussion' to  be  eitker  a  mathematical  equation  or  a  set  of  simultaneous  mathe¬ 
matical  equations  which  doscribo  tho  behavior  of  a  real  or  concoptuul  physical 
system.  Some  examples  of  such  models  are: 

Y  ■  a  +  bX  +  cX2  (1) 

and 

Y  -  a'eb'X  (2) 


where  Y  is  the  muzzle  velocity  of  a  projectile  launched  from  a  gun  with  a 
propellant  charge  weight  of  X  and  a,  b,  c,  a*  and  b'  are  parameters  depending 
on  other  ballistic  conditions. 

Equations  (1)  and  (2)  form  an  example  of  two  different  models  which  may 
be  used  to  describe  the  same  phenomenon.  If  you  were  to  fire  a  number  of 
rounds,  systematically  varying  propellant  charge  weight  and  recording  charge 
weight  and  muzzle  velocity  for  each  round  fired,  you  could  fit  either  equation 
Cl)  or  equation  (2)  to  your  data,  i.e.,  you  could  find  values  for  the  unknown 
parameters  a,  b,  c,  a*  and  b'  which  bost  predict  the  observed  Y's  using 
the  observed  X's.  The  next  natural  step  is  to  use  one  of  the  fitted  equations 
to  predict  tho  muzzle  velocity  of  a  projectile  fired  with  a  propellant  charge 
weight  of  X.  This  prediction  would  contain  three  sources  of  error. 

One  source  of  prediction  error  is  the  systematic  error  resulting  from 
the  inexact  structure  of  the  model  used.  Clearly  predictions  given  by  equa¬ 
tion  (1)  will  differ  from  those  given  by  equation  (2)  and  one  can  never  be 
certain  which  of  the  two  is  the  most  reasonable  to  use  or  even  whether  some 
other  functional  form  of  the  model  would  be  appropriate.  Thif  type  of  error 
seems  to  be  best  controlled  by  careful  consideration  of  the  physical  prin¬ 
ciples  underlying  the  process  to  be  modeled  and  recourse  to  sound  engineering 
judgements.  This  type  of  error  propagation  will  not  be  considered  further  in 
our  discussion. 

The  other  two  errors  which  manifest  themselves  are  random  variables. 

They  are  the  errors  in  estimating  the  parameters  in  the  model  and  the  inexact 
value  of  the  independent  variable (s)  from  which  predictions  are  to  be  made. 
These  are  the  errors  which  will  be  discussed  below. 
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For  a  good  treatment  of  the  problem  of  propagation  of  errors  in  the 
model  input  (independent)  variables  the  reader  is  referred  to  references  1  and  2. 

We  do  not  propose  to  give  a  collection  of  recipes  for  error  analysis  for 
physical  models.  Rather,  we  will  discuss  several  techniques  and  some  of 
their  implications  under  usual  assumptions. 


2.  ONE  FUNCTION  OF  ONE  RANDOM  VARIABLE.  Consider  the  physical  model 

Y  -  g(X)  (3) 

where  X  is  a  random  variable  with  distribution  function  P(x;6)  with  0  boing 
a  vector  of  one  or  more  parameters.  Since  X  is  a  random  variable  it  follows 
that  Y  is  also  a  random  variable  and  wo  aro  interested  in  the  properties  of 
Y.  The  question  most  often  asked  is,  "What  are  the  mean  and  variance  of  Y?" 

The  approximate  answer  to  this  question  is  usually  obtained  by  expanding 
g(X)  in  a  Taylor  sorios  about  B(X),  truncating  tho  sorios  appropriately  and 
taking  the  expected  value  of  the  truncated  series  with  rospect  to  the  random 

variable  X.1'2 

In  the  case  of  one  independent  variable,  if  derivatives  of  all  order 
exist,  it  follows  that 

Y  *  l  gCr)(y)(X-y)r  (4) 

XmO 

where  B(X)  ■  y  and  g^(y)  is  the  rth  derivative  of  g(X)  evaluated  at  the 
point  X  ■  y. 

If  we  define  ur  »  B(X-y)*;  r  ■  0,  1,  2,  we  can  write 

60 

E(Y)  -  l  k  g(r3(y)y_.  (5) 

r-0  11  r 

It  then  follows  that 

Y  -  B(Y)  -  l  k  gCr)(y)[(X-y)r-yJ 
r**0  *  * 
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and 


00  00 

Var(Y)  -  E[Y-E(Y)]2  -  £  [  1  i.  g(r) (y)g(s) Cy)  E[(x-y)r-yJ  [(x-u)#-uj 

r-0  s-0  5  r  5 

'  J>  ,Ii  ®  *i  |Wmi(,1miv.  •  v,i  “> 

since  yQ  ■  1  and  y^  ■  0. 

Equations  (S)  and  (6)  give  the  nean  and  variunce  of  Y  ■  g(X)  in  terms  of  the 
higher  moments  of  X.  Similar  relationships  can  be  easily  derived  for  the 
higher  moments  of  Y. 

Por  purpose  of  illustration,  let  us  apply  equations  (S)  and  (6)  to  find 
the  mean  and  variance  of  the  quadratic  model  of  equation  (1) . 

Y  -  g(X)  «  a  bX  +  cX2 
g'(X)  -  b  +  2cX 
g"(X)  ■  2c 

g(ri(X)  *  0  for  r  >  3 

It  follows  from  equation  (5)  that 

B(Y)  ■  a  +  by  +  cy2  +  ccr2  (7) 

2 

where  o  ■  y2  is  the  variance  of  X.  The  usual  estimate  of  the  expected 

value  of  Y  is  given  by  B(Y)  m  g(y)  ■  a  *  by  +  cy2.  This  estimate  is  in  error 
2 

by  co  ,  which  may  or  may  not  be  negligible. 

It  follows  from  equation  (6)  that 

Var(Y)  ■  (b+2cy)2c2  +  2c(b+2cy)y3  ♦  c2(y4-o*)  (8) 

so  that  the  usual  estimate  of  the  variance  of  Y,  Var(Y)  •*  [g'(u)]2a2  ■ 

(b  +  2cy)2a2  is  in  error  by  2c(b  +  2cy)y3  +  c2(y4  -  a*).  If  the  distribution 
of  X  is  symmetric  about  y,  then  y3  ■  0  and  the  error  reduces  to  c2(y4  -  o*). 

Taking  our  analysis  one  step  further,  we  note  that  if  X  has  a  normal  distri¬ 
bution,  or  approximately  su  via  the  central  limit  theorem,  then 
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;  r  odd 


Mr  -  E(X-y)r  -  0 


qrrl 

2r^2(r/2)l 


r  even 


2  2  2  4  2  4 

and  Var(Y)  ■  (b  ♦  2ey)  o  +  2c  a  so  the  error  becomes  2c  a  . 

No  next  consider  the  usual  case  in  which  the  parameters  a,  b  and  c  are 
random  variables.  This  occurs  when  observations  are  made  on  X  and  Y  and 
these  observations  are  used  to  estimate  the  parameters. 


Consider  the  random  vector  Ca,b,c) '  which  is  independent  of  the  random 
variable  X  (tho  input  to  our  predictive  modol) .  Lot 


and 


B[(a,b,c)']  -  (A,B,C) * 


Cov[(afb,c) '] 


If  we  use  least  squares  to  fit  the  parameters  to  the  data  and  assume  indepen* 
dent  normally  distributed  residuals,  then  we  can  calculate  good  estimates 
of  the  vector  of  means  and  the  covariance  matrix. 

In  this  case 


B(Y)  ■  E(a  +  bX  ♦  cX2)  ■  A  +  By  +  C(o2  ♦  p2)  ■  A  +  By 


+  CO2  +  Cm2, 


(9! 


which  is  precisely  the  same  as  when  the  parameters  were  not  random  variables 
provided  that  they  are  unbiased  estimates  of  the  true  coefficients. 

Similarly 


YarCY)  -  oj  +  o2  Yar(X)  +  o2  VarCX2) 

+  2  Cov(a,  bX)  +  2  Cov(a,  cX2) 

+  2  CovCbX,  cX2) 

•  *  °b  °2  *  °c  VtrCx2)  *  covariance  terms. 
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The  covariance  terms  in  the  above  relationship  are  calculated  as  follows: 
Cov(a,  bX)  -  B(abX)  -  E(a)B(bX)  •  yoab 

Cov(a,  cX2)  •  E(acX2)  -  B(a)B(cX2)  ■  cmAu2  +  a2) 

1C 

Cov(bX,  cX2)  -  B(bcX3)  -  E(bX)B(cX2) 

•  0bc(M3  ♦  Sycr2  «■  VI3)  +  BCu3  *  2BCW02. 
Substituting  these  expressions  in  the  above  relationship  leads  to 

Var(Y)  •  a2  *  a2o2  ♦  c2(p4  ♦  4yii3  +  4y2o2  -  o4)  ♦  2yoob 

4  2  °ac^2  +  +  2  °bc^w3  4  ♦  n3) 

+  2  BC(y3  ♦  2  y<72),  (10) 

which  is  quite  different  from  the  usual  estimate. 

Equations  of  the  form  of  equation  (10)  should  prove  useful  for  determining 
sample  sisos  required  for  experiments  to  collect  data  for  estimation  of  the 
parameters  of  a  mathematical  model  which  will  be  used  for  predictive  purposos. 

3-  TWO  OR  WORE  FUNCTIONS  OF  ONE  RANDOM  VARIABLE. 

Let  Y1  ■  gj(X)  and 

*2  »  g2(X) 

e 

s 

e 

vk  ■  *k<»- 

that  is.  we  have  k  >  2  functions  of  one  random  variable. 
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In  this  case  we  use  equations  (S)  and  (6)  to  find  B(Y^)  and  Var(Y^) 
for  all  i  ■  1,  2,  ...,k.  The  only  undetermined  moments  are  Cov(Yi#  Yj),  i  j*  j. 

We  note  that 

Oft 

Y,  -  B(Y.)  -  l  g,Cr)  (y)[CX-y)r  -  V  J  and 
*  *  r»0  *  *  * 

00 

y,  ■  B(yJ  -  l  i,  g,(s)  CvH(x-u)*  -  ygl. 

J  J  s*0  J 

Thus 

Cov(Yif  Yj)  -  E[  Yt  -  BCYpifYj  -  ECYj)] 

00 

-  r  £  0  rTsT  gir)  Cw)gjC5)(y)B[(X-y)r+#-  yr(X-y)s-  Ms(X-u)r+  yyu 

r  s-0  g*  ^^t^r+s  “  V.]* 

If  we  recall  that  u0  ■  B(X-u)°  ■  1  and  ■  B(X-y)  ■  0,  we  see  that  for  r  •  0 
or  s  ■  0  u  -  u  y  ■  0  so  we  can  write 

j  x  o 

CovtW  '  r  I.J  rTTT  «ir)(w)*J(,)Cii)[|irM  -  Urw,].  (11) 


4.  ONE  FUNCTION  OP  TWO  OR  MORE  RANDOM  VARIABLES.  The  problem  >£  one 
function  of  two  or  more  random  variables  is  treated  completely  analogously  to 
that  of  one  function  of  one  random  variable.  That  is,  we  expand  the  function 
in  a  Taylor  series  about  the  vector  of  means  of  the  random  input  vector  and 
truncate  appropriately.  Before  proceeding  with  this,  we  state  the  following 
theorem  which  can  be  found  in  reference  3; 

Theorem  6.2  (Apostle!) 

Let  g  have  continuous  partial  derivatives  of  order  m  at  each  point  of 
an  open  set  S  of  Bft.  If  a  and  b  are  both  elements  of  S,  a  ^  b  and  the  line 

segment  joining  a  and  b  lies  in  S,  then  there  exists  a  point  a  on  the  line 
segment  L(a,b)  such  that 


where ; 


F 


ra-1 


gW  -  8(b)  -  I  dkg(a;b-a)  ♦  ±,  dBg(z;b-a) 


1  .B. 


k-0 


a  ■  (a^ ,  #2  •  •  • »  1  > 


b  ■  (bi ,  b9,  •  •«,  b^) 1 


'1*  “2* 
Xji 


x  -  CX1#  x„  ....  xn)', 


n  n 


d  g(a;b-a)  ■  j[  J.  •••  [  D.  ,  .  g(a)  t,  t.  .  ..t. 

i,«l  i.»l  XV2“'\  *2  H 


■  b.  -  a. 

ij 


“*  “iji;,...!,.  ■  5T1‘5Sq^...Sx1 


Me  will  uae  the  following  theorem  from  reference  3  to  simplify  expressions 
in  our  series  expansions. 

Theorem:  If  D.jg(X),  Djg(X)  and  D^g(X)  are  continuous  in  a  neighborhood 
of  the  point  (X^  Xj)  in  B2,  then  Djj8(xi*X2^  exists  and  gCX^Xj)  ■ 

Before  using  the  relationships  given  in  this  section  one  should  verify 
that  the  function  g(X)  satisfies  the  conditions  of  the  above  two  theorems. 

We  consider  the  case  where  n  ■  2,  that  is,  Y  »  g(X)  is  a  function  of  two 
random  variables. 


and 


Let  X  - 

cxx,  x2r 

2 

Cov(X)  ■ 

°1  °12 
2 

012  °2 

E(X)  -  ( Vv  V2)' 
and  define 


V*  .  -  B[(X1-y,)r(X,-p-)a]. 


Expanding  g(X)  about  the  point  X  ■  y  in  e  Taylor  series  leads  to 


Y  ■  g(X)  •>  g(X1#X2)  •  g(y)  +  J  dkg  (y;X-y) 

where  we  neglect  the  remainder  tern,  R  ■  — j  d"g(z;X-y),  z  e  L(X,y). 
ducing  the  notation 


Intro- 


,  (w .  ?r.*«csL 

r»*  axj  3xJ  X-y 

and  using  the  fact  that  g_  _(y)  ■  g,  „(y) ,  our  expansion  bocomos 

*#•  »#r 

Y  ■  ,tX>  :  X  *'  .1  '  *  tl2) 


The  expected  value  of  Y  is  approximately 


HCY)  -  B[g(X)]  i  *f  k  l  (  J  )ga  r_(y) [y.  ] 

r-0  rl  a-0  *  a,r  *  *‘r  ** 


Squaring  equation  C12)  gives 


r’  ■  X  X  *  *  .I  ,i<  •  u‘),*vtx*  •  ^ 


r+u-s-v 


and  it  follows  that 


a-1  ari 


B(Y2)  "  Jo  u-0  7lCTa-0  v-0  C  ^  C  v  J  g*»»-*Cw5gv#u-v(M3Ms+v,r^u-s-v  C14) 


Squaring  equation  (13)  and  subtracting  the  result  froa  equation  (14)  leads 


Var(Y)  -  B(Y2)  -  [B(Y))2 


"  Ji  1  »H»r  .L  X  C  »  )(  v  )gs,r-s(w)gv,u-vCu) 


r«l  u»l  a»0  v»0 


^s+v,r+u-s-v  “  ^Sjr-a^v.u-v^ 
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To  illustrate  the  use  of  the  above,  we  consider  the  case  in  which  X  has 
a  bivariate  normal  distribution.  Here,  yr+s  »  0  for  r+s  odd,  and  all  other 

2  2 

values  of  yy+s  depend  only  on  y^,  y2»  <*2  *IM*  °12*  In  CMe 

ECY)  -g(U)  +  2«20Cw)Ol  *  2*llM°12  4  W(l°C2 

+  t  *40(y)fll  4  7  g31(w)ol°12  4  T  g22(w)t°l°2  4 
+  7  *ljC^°2°12  4  I  *04Cli3o2  *  (l6) 


and 

Var(Y)  i  aj[g10(y)2]  ♦  <*i2t2g10Cy)g0l(y)l  4  °2t*oiCu32i 
+  oJUio^)*30^  4  5«20Cw)2j 
*  °2t«01C^*0S(y)  4  3*02  ^ 

+  <*20^t«ol(v)g21(u)  ♦  g10(y)g12Cvi)  ♦  gjjCy)2] 

4  ol°12[*01Cw5g,3<)M  4  s«10Cw5*21^  4  2*llMg20(u^ 

4  °2°12t»10(w)g03C^  4  Sg01(u)g12Cy)  4  2gli(u)g02CM)3 

4  °12l2g01Cvi)g21Cvi3  4  2gloMg12M  4  g02(wJg20Cw5  4  gllCu)21  (17) 

Though  equations  (16)  and  (17)  may  be  lengthy,  they  are  straightforward 
and  can  be  easily  evaluated  on  a  computer. 

Equations  (13)  and  (IS)  readily  generalize  to  the  case  of  a  function  of 
k  >  2  variables.  Since  the  extended  equations  are  messy  and  notationally  complex, 
this  effort  will  be  delegated  as  a  chore  for  the  reader. 
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5.  SEVERAL  FUNCTIONS  OP  MORE  THAN  ONE  VARIABLE. 


Let  Y  -  [Yj,  Y2,  ....  Yp] * 

•  [gCi)CX),  g(2)(X),  ....  g(p)(X))‘ 


where  X  is  e  random  vector  with  properties  given  in  the  previous  section 
and  Yr  ■  g^(X).  Equations  (IS)  and  (15)  give  approximations  for  B(Y^)  and 
Vor(Yj).  It  romains  for  us  to  find  Cov(Yr,  Yu). 

The  argument  that  was  used  to  derive  equation  (15)  leads  immediately  to 

Cov(Y  ,Y  )i"f  Y  ^  l  I  (  I  K  J  )g#Cr)r-s(p)«iULCw)  C18) 

r  u  r»l  u»l  u‘  a«0  v-0  *  v  1,r  *  v,u-v 

^s+v,r*u-s-v  "  ws,r-spv,u-v^ * 


6.  SUMMARY  AND  CONCLUSIONS.  Error  propagation  in  mathematical  modols 
is  simply  another  name  for  the  study  of  one  or  more  functions  of  one  or 
more  random  variables.  The  ideal  treatment  of  this  problem  is  to  derive  the 
actual  joint  distribution  of  the  new  set  of  random  variables  which  are  trans¬ 
formations  of  the  original  set.  Unfortunately  this  problem  is  unsolved  for 

all  but  a  relatively  few  commonly  used  functions. 

» 

Common  practice  for  these  unsolved  problems  is  to  approximate  the  means, 
variances  and  covariances  of  the  new  variables  and  use  these  approximations 
to  construct  error  bars. 

We  have  given  procedures  for  refining  the  most  commonly  used  approxima¬ 
tions.  Our  equations  become  a  bit  awkward  for  random  vectors  of  high  dimension 
but  they  remain  straightforward  and  manageable  on  a  computer. 

The  procedures  given  herein  should  be  useful  for  evaluating  approximations 
currently  in  use  and  they  promise  to  furnish  a  vehicle  for  developing  improved 
approximations  for  special  classes  of  functions,  e.g.,  convex  functions. 
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hiii  >1  mriro  umiHmintu' 


On  R«Qcn«rstiv»  ProctsMs 
In  Dimer  art.*  Timm 

Halter  L«  Smith 

University  of  North  Carolina,  Chapel  Hill 


In  thia  paper  we  ahall  praaent  a  aomewhat  hauriatlc  account  of  reaulta  we  have  been 

obtaining  which  we  hope  will  prove  to  be  of  value  in  applying  probabiliatic  ideaa  to 

many  atochaatic  problem*  which  may  arise  in  practice. 

He  begin  by  explaining  the  idea  of  a  renegerative  atochaatic  process,  First  we 

must  understand  what  is  meant  by  a  fcpUD  a  typical  one  of  which  we  ahall  denote  T, 

For  the  present  discussion,  in  which  we  shall  be  confining  ourselves  to  events 

occurring  on  a  discrete  time  scale,  T  will  consist  of 

(a)  An  integer  X  called  the  duration  of  the  tour, 

<b>  A  real-valued  function  T(t)  defined  for  t«0,l,2t„„Xi  this  is  called  the 
oraoh  of  the  tour, 

(c)  A  final  value  or  residue,  denoted  Yi  which  can  be  regarded  as  T(X), 

He  imagine  a  population  of  tours,  from  which  repeated  sampling  will  produce  an 
lid  sequence  Tj,  T2*m*  and  so  on,  The  successive  durations  are  X^,  X^„»,  and  it 
is  clear  that  they  form  a  recurrent  event  process  on  the  discrete  time  scale.  He 
call  this  recurrent  event  process  the  gig  for  the  cumulative  process,  Let  us  denote 
the  probability  distribution  of  the  CXn>  by  fj,  f2»  etc,,  noting  that  we  shall 
always  assume  that  FCXn  ■  0>  ■  0,  He  can  then  call  into  play  the  pgf  F(x)  ■ 

2^  xnfn»  which  will  converge  for  small  enough  |xl»  For  the  present  exposition  we 


PHIQIOUn  Z*0B  H  AMT -MW  H1MD 
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shall  assume  there  is  *  O  1  such  that  F(5)  <  do*  This  Mill  snsurs  that  F(z)  is 
analytic  in  the  open  disk  |z|  <  ?. 

He  then  build  up  a  process  on  the  infinite  discrete  time  scale  by  concatenating 
the  tours  X2*  ....  Let  us  write  n  "  xi  +  *2  +  *  xn  for  th* tim*  At  which 

Xn+1  begins  (so  that  BQ  ■  0).  Then  if  Sn  £  t  <  Sn+1  we  can  define  the  value  of  the 
regenerative  process  at  t  by  Tn<  t  -  8n)»  where  Tn  stands  for  the  graph  of  Xn*  In 
this  conection  it  will  be  convenient  to  write  N(t)  for  the  largest  integer  k  for 
which  £  t.  He  shall  then  write  T*(.)  for  the  graph  of  (which  is  the  tour 

oporating  at  time  t).  Similarly  we  shall  write  S#(t>  for 

It  will  also  help  if  ws  write  Yj  for  the  residue  of  Xj.  He  can  then  define  a 
cumulative  process  W(t>  as  follows* 

H(t)  ■  Sj  ^  N(t)  +  T*  (t  -  8*<t». 

In  words*  H<t)  is  the  sum  of  all  residues  of  tours  which  ended  prior  to  time  t, 
together  with  the  current  value  of  the  regenerative  process  (as  described  above). 

In  all  really  interesting  applications*  Xj  and  Yj  are  og£  independent)  of 
course*  the  very  nature  of  the  underlying  sampling  procedure  ensures  that  the 
successive  vectors  C  Xn*Yn>  m  independent. 

The  main  question  which  we  try  to  answer  in  a  study  of  such  cumulative 
processes  concerns  the  asymptotic  behavior  of  W(t)*  as  t  grows  without  bound.  It  it 
possible  to  define  multivariate  versions  of  all  that  we  have  described*  and  it  ii 
possible  to  obtain  results  in  the  more  general  case*  but  we  shal  eschew  this 
generality  in  the  present  exposition. 

A  very  useful  tool  in  tackling  H(t)  is  the  characteristic  function,  defined  for 
real  values  of  the  dummy  8*  by  the  equation 
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™  EoxpCiS  W(t)3 


It  then  turn*  out  that  on*  nsadi  th*  following  function* 

*(8)  -  Eexp  Ci8Y() 

X  1 

Q(z>8)  -  Ex  1  tip  LiBYjl 

R(z,e)  ■  E  S*  *xp  Ci8T|(m)3t 
where  th*  sum  S*  extend*  ovar  tha  ranga  m^Otom^Xj-i* 

Tha  undarlying  modal  should)  in  many  casa*)  allow  tha  calculation  of  thaaa 
various  functions  wo  Hava  just  defined)  Howovor»  tha  datsrminatlon  of  <|>(8»t)  is 
difficult)  What  it  possible  is  to  determine  tha  generating  function 

<p°l8>x)  -  cp(6(t), 

A  certain  amount  of  fairly  straightforward  reasoning  will  than  lead  to  tha  neat 
result 

4>°<8)Z>  -  R'Z)8)/C  1  -  G (Xt6)3 
There  are  two  spedal  cases  of  particular  interest) 

CLfcSfi  A 

In  this  class  it  is  supposed  that  T{tt) » t  for  all  0&t£Xj»  This  corresponds 
to  tha  situation  where  Y,  is  a  cost  interred  by  tha  tour)  arm  this  cost  has  to  be 
paid  at  tha  instant  th*  tour  begins* 

For  a  Class  A  process  it  is  easy  to  show  that 

R(z,8>  -  Ctii<8)-GU)6>3/Cl-z] 


CLABS  B 

Hera  it  is  assumed  that  TJt)  ■  0  for  all  t  C  Xt>  although  it  may  well  be  that 
Y,  differs  from  0»  This  corresponds  to  the  situation  when  there  is  a  payment  for  a 
tdur)  but  it  is  only  made  at  the  toA  of  that  tour* 

This  case  give*  a  particularly  simple  result 

r(Zi8)  «  £1  -?(x)3/Cl  -xl 

A  Class  B  process  of  great  importance)  and  much  studied  in  its  own  right) 
arises  when  every  r'esidue  has  the  value  unity*  In  this  case  the  cumulative  process 
Wit)  *  Kit)*  and  is  the  "renewal  count)" 


For  the  renewal  count  proceee  on*  can  ehow  that 

«p°(0,z>  -  Cl  -  FdlJ/Cl-xJCl  -  •ieF(xH, 

It  traneplree  that  on t  con  prove  on  identity  for  Cloee  A  proceeeee*  which 
porallele  doeely  the  femoue  Fundamental  Identity  of  Abraham  Hold  in  toquontiol 
onalyeie*  Actually  the  argument  here  it  much  eoeier  then  it  ie  in  eequential 
onalyeie*  The  identity  we  con  get  ie 

,i6W(t) 

E  tiwenNrt>+l 

Unfortunately  there  ie  no  correepondingly  elegant  reeult  for  the  Cloee  B  prooeee 
(ori  indeed*  the  general  cumulative  proceee) 

A  trick  that  M.B*Bartlett  expoeed  in  hie  contribution  to  a  8ympoelum 
on  Btochaetic  Proceeeee  (Bartlett*  19491  can  be  ueed  to  extract  infdrmation  from 
thie  Fundamental  Identity*  It  conelete  in  looking  for  a  C  4  0  for  which  4i(C  >  ■  1* 

For  example*  if  we  euppoee  that  <i<0)  ■  <XA  - 10)2  we  find  that  C  ■  -211,  and  are  led 
from  the  Fundamental  Identity  to  the  reeult 

E.2XH<t>  .  if 


from  which  it  ie  eaey  to  infer  the  inequality 


PCH(t)  l  x>  £,  e-2Xx. 


To  eee  how  eaeily  the  identity  may  be  derived*  let  ue  chooee  any  •  euch  that 
<i(01  if  0*  and  eet 

log  <1(61 


%  -  Yr 


10 
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If  we  use  Yj  in  place  of  Yj  in  tha  formula  for  the  Class  A  process*  than  wa  find 
that 

Ste)-S<x»8> 

Q>°(8,x)  m  - - -  , 

a-i)a-3(x,en 

But  it  i»  a  aimpla  mattar  to  show  that  $(6)  *  1*  from  which  it  foUowa  at 
onca  that<i»0(Q,x)  ■  i/U-x)*  Thia  obvioualy  impliaa  that  0X8, t>  ■  1  for  all  t  * 

0,1,2,.*..  Tha  darivation  of  tha  Fundamental  Identity  ia  now  a  aimpla  matter . 

tat  ua  now  conaider  a  particular  example  of  the  general  case  which  la  neither  a 
Claaa  A  nor  B,  Suppoae  that  each  raaidue  Y  ia  N  (0»  1+X).  Thia  repreaenta  a 
aituation  where  there  ia  a  unit  expected  eet-up  coat  for  each  tour  and  then  a  random 
extra  coat  whoae  expectation!  conditional  on  the  duration  X  of  the  tour,  ia  X.  For 
thia  caae  we  can  aee  that 

CKx,9>  -  e"®2/2  Fixe"^2). 

To  underatand  the  behavior  of  4>(9,t)  we  need  to  look  at  tha  eingularitioo  in 
the  complex  x-plena  of  <D°l0,xh  Theae  plainly  occur  where  OU»8)  ■  1,  and*  in  the 
preaent  apodal  example*  thia  ia  where 

(1)  Flxe"®a/2>  *  e®2 

Let  ua  chooaa  a  amall  f  >  0  and  aaaume  that  tit  £  f*  Then  <l)  will  have  a 
aolution  x  ■  C(i>,  aay*  which  tenda  to  xero  aa  9  tanda  to  xaro,  end  there  will  be  a 
€  >  i  auch  that  c  (6)  ia  tha  an 1%  xero  of  i  -  G(x,0)  in  the  drde  lx|  £  €* 

To  carry  the  example  furtheri  let  ua  be  apedfic  about  the  distribution  of  the 
durations  Xn*  Let  u»  suppose  that  Xn  has  a  geometric  distribution  auch  with  pgf 


F(z)  ■  pzA  1  -  qz)« 


Than  we  shall  find  that 


-ep/2 

,♦  pe  9 


Notice  that  that  thia  formula  show*  that  4(8)  ->les0-*Oand-»<Daa8->®. 

Our  purpose  in  examining  this  very  special  example  is  to  show  the  sort  of 
behavior  to  be  expected  in  the  general  situation*  Indeed*  careful  complex  analysis 
shows  this  sort  of  behavior  to  be  a  feature  of  the  general  easel  but  it  is  felt 
that*  for  a  simple  exposition  such  as  the  present*  an  illustration  using  a  special 
case  will  be  more  acceptable  to  the  reader* 


Let  us  now  resume  the  discussion  of  the  most  general  situation  and  let  Gj  be 
the  circle  Izl  ■  8  >  I  U8)|*  and  let  be  the  circle  III »  p  <  I  c<8)|*  interior  to 
Cj.  Then  by  the  usual  function-theory  argument  we  are  led  to  the  result 


0<6*t)  ■ 


Suppose  we  now  allow  p  to  grow  so  that  the  contour  Cj  passes  across  the  pole  at 
CIS)*  In  doing  this  we  shall  increase  the  value  of  the  contour  integral  by  the 
residue  X*  say*  of  the  function 


at  the  pole  at  5  (8)«  It  then  apppears  that  we  shall  have 
<2>  4»(8*t)  ■  -X  ♦  etc. 
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But  it  is  not  too  difficult  to  show  that  the  integral  in  (2)  it  Q(€~*)t  this 
integral  Mill  thus  go  to  xero  pnuch  fatter  than  any  ttrma  we  shall  retain*  and  to 
from  now  on  wo  shall  ignor.e  it*  The  remaining  problem  it  to  determine  the  value  of 
Xi  which*  of  course*  will  be  a  function  of  t*  Let  us  write  Oz<C  <B>»01  to  denote 

t  feowm, .  t„, 


Then  we  shall  find  that 

<P(d*t)  -  of^ShS‘c;<e,3’<^1>  4  negligible  terms 

At  this  stage  it  it  convenient  to  introduce  a  function  A(0)  with  the  property 
that  c<0>  ■  exp  A(i)|  thus  A<0)  -B  0  at  0  -B  0*  This  function  A(0)  it  very  important* 
and  to  we  shall  give  it  a  careful  examination*  Xt  it  possible  to  show  that  it  will 
have  a  Taylor  series  expansion 

C  A(ft)|ji0+  (i01|jQi  ]  ♦  U/aUCaW)2^  +  2t(SH18>pu  ♦  (i0>2pO2l  ♦  etc  •  0 

Xt  it  possible*  though  with  increasing  complexity*  to  compute  the  <An>  in 
terms  of  "known"  moments*  The  calculation  is  based  on  the  equation 

G<eA(0>,8>  •  1* 

and  this  leads  to  the  following  relation* 

,t  A(I1jj10  if  <itty01J  +  a/2>CCAtB»2p20  +  2A(«Ui8>|fn  +  (ill2}^!  ♦  etc  •  0 

There  teems  no  way  of  escaping  a  considerable  amount  of  tedious  computation  at 
this  point*  One  must  equate  coefficients  of  powers  of  B  to  sero  and  determine 
the  <An>  from  the  resulting  equations*  For  n  >  5  the  formulae  for  these  CAn) 
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become  vary  large*  Kara  wa  shall  content  ouraalvaa  with  quoting  tha  only  tha 
formula*  for  Aj  and  Thaaa  ara  at  follows 


A 


1 


to 

PlO 


k  mm  to  m  toUtot  .  itoto 

Pio  (pl0)3  <p10>2 

Lat  us  now  sot 

t?  ■  P  (Y  -  X  t ' 

*  i  ho  i 


Than  it  is  possibla  to  show  that  a  much  simpler  formula  exists  for  A^, 


In  terms  of  the  quantities  we  have  now  introduced  we  thus  have  the  equation 


E,iew<t)  .  j  j  .-(t+DCAjUO)  ♦  <i/2)A2(i*)2  +  .*.  3 

s 

and  this  will  hold  for  all  |8|  £ 8*  for  soma  small  J  >  0. 

But  for  iqx  6i  if  t  is  large  enough*  wa  shall  find  that 

IttStt  I  * 9 

Thus*  if  wa  replace  0  by  •/✓(t+O*  we  shall  find  that 
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it  asymptotically  N<0,t?/p10>. 

This  Central  Limit  Theorem  it  useful,  It  can  be,  and  hat  been,  proved  without 
tht  details  in  which  we  have  become  embroiled!  but  these  details  are  necessary  if  w. 

want  to  obtain  the  more  delicate  results. 

With  a  great  deal  of  tedious  computation,  one  can  obtain  Edgeworth-type 
improvements,  in  the  form  of  series  expansions,  in  place  of  the  single  dominant 
normal  density  term,  Evidently  we  can  write 

_  .  _  _,X  JOY 

QJiiO)  ■  Ex  e  , 
z 

so  that 

e>  -  |j10  ♦  <ie>^i<p2o“ho>  +  hi3  *  ,tCm 
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However*  once  one  asks  for  more  than  the  dominant  term*  the  function  R(e  »8> 
has  to  be  taken  into  consideration!  In  this  connection  some  curious  moment-like 
numbers 

_Xfl 

p„  *  e  i  ,:0  ir<T,ii»* 

emergei  They  should  be  determinable  from  the  known  specification  of  the  underlying 
modeli  If  one  makes  use  of  these  numbers  one  is  led  to  an  expansion  like  the 
following  (which  we  quote  to  only  one  term  beyond  the  dominant  one) 


c  G1(C(e/v<t+i»,e/v<t+i> ) 


(19)C 


i  + 


v<tn> 


* 


Of  course*  nothing  but  hard  work  prevent  ones  from  obtaining  as  many  extra  terms  in 
this  expansion  as  one  wishes!  In  the  above  abbreviated  expansion 


c  .  .Hi  *  MSI  +  £si, 

ho  2(ho>2  2ho  ho 

If  we  now  write  v  in  place  of 't2/^*  we  are  thus  led  to  the  result 

E  ie*<t>  _  _  A3(-i9)*  0(l/t){ 

E#  “*  J  1  +nM+1)  6^(t+i>  *  0(1/t)V 


Prom  this  last  result  one  can  thus  infer  that  asymptotic  probability  statements 
about  Z(t)  will  be  better  if*  instead  of  a  simple  normal  density*  we  use  the 
"density" 
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where 


-x^/2v  ( 
✓(2nv>  t 


1  + 


! 

v<t+i>  +mr 


Cj  •  vC0  ♦  <l/2)AgV3, 

Cg  -  <t/6)AgV3 


Zn  deriving  this  result  we  have  used  the  feet  that  multiplying  *  Fourier 
Transform  by  (10)  corresponds  te  differentiating  the  original  function  of  x*  with 
repsect  to  x> 

It  is  possible  to  extend  all  the  ideas  we  have  discussed*  end  derive  roughly 
similar  results*  when  we  replace  the  simple  renewal-type  MbedM  of  the  cumulative 
process  with  a  "Semi-Markov"  one*  At  the  conference*  some  brief  discussion  was 
gi'  an  of  this  latter  more  general  model*  However*  we  shell  not  discuss  it  here* 
preferring  to  delay  an  exposition  until  somewhat  more  comprehensive  results  have 
been  obtained* 
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